Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jukebox model (replaces #16875) #17826

Merged
merged 225 commits into from Nov 10, 2022
Merged
Show file tree
Hide file tree
Changes from 221 commits
Commits
Show all changes
225 commits
Select commit Hold shift + click to select a range
b92a42a
Clean historuy
ArthurZucker Jun 22, 2022
d316211
delete tf related function
ArthurZucker Jun 23, 2022
0263507
fix copies
ArthurZucker Jun 23, 2022
4f67909
update
ArthurZucker Jun 23, 2022
20cee2e
style and delete fast tokenzer
ArthurZucker Jun 23, 2022
a3dace0
fix consistency check
ArthurZucker Jun 24, 2022
8ba643d
update test and modelling
ArthurZucker Jun 24, 2022
93ae4fe
add progress bar, `rich`dependency
ArthurZucker Jun 26, 2022
684af17
clean and add progress bars
ArthurZucker Jun 27, 2022
29d2614
update
ArthurZucker Jun 30, 2022
81afaad
update code
ArthurZucker Jun 30, 2022
7d361d8
update tokenizer doc
ArthurZucker Jun 30, 2022
1d4d0d2
only CPU run for now need to clean and handle device properly
ArthurZucker Jul 1, 2022
965b2dc
update tokenizer to support v3 dictionnary
ArthurZucker Jul 4, 2022
df30e00
update tests
ArthurZucker Jul 4, 2022
6c761c1
style
ArthurZucker Jul 4, 2022
adc849d
clean test
ArthurZucker Jul 4, 2022
1e5a94e
fix tokenizer
ArthurZucker Jul 4, 2022
0450a37
1b expected outputs and major update
ArthurZucker Jul 4, 2022
67d2c68
refactoir toeknizer
ArthurZucker Jul 5, 2022
4b9fd66
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Jul 5, 2022
36dee3e
update
ArthurZucker Jul 5, 2022
4b56ed9
fix tokenization
ArthurZucker Jul 6, 2022
92bed83
style
ArthurZucker Jul 6, 2022
3bfc36c
major refactoring
ArthurZucker Jul 6, 2022
3ff1468
fix 1b and 5b, refactor tokenizer. Both models are ready
ArthurZucker Jul 8, 2022
58cb7bd
Add slow GPU tests that needs to be done later on
ArthurZucker Jul 8, 2022
4c90a40
style
ArthurZucker Jul 8, 2022
3c90c36
style
ArthurZucker Jul 8, 2022
b5a0a2e
style and change
ArthurZucker Jul 8, 2022
e7155ce
quality check
ArthurZucker Jul 8, 2022
f68b09e
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Jul 8, 2022
0c66dd1
simplify music generation
ArthurZucker Jul 19, 2022
30b2084
clean tests
ArthurZucker Jul 19, 2022
33aba3b
style
ArthurZucker Jul 19, 2022
cdcd5e2
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Jul 19, 2022
6d568b0
fix tests
ArthurZucker Jul 19, 2022
81bc0df
small fix on VQVAE encoding
ArthurZucker Jul 20, 2022
9d0f7e3
Merge branch 'huggingface:main' into jukebox
ArthurZucker Jul 20, 2022
92acc77
notebook for generation
ArthurZucker Jul 20, 2022
b982798
quality
ArthurZucker Jul 20, 2022
d6f8807
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Jul 20, 2022
fe0b96d
Merge branch 'huggingface:main' into jukebox
ArthurZucker Jul 20, 2022
10688a4
update sampling
ArthurZucker Jul 20, 2022
d7e1464
fix
ArthurZucker Jul 20, 2022
5fabaa7
fix 1b tokenizer
ArthurZucker Jul 20, 2022
1a593d9
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Jul 21, 2022
3004d3c
clean modeling
ArthurZucker Jul 21, 2022
5058108
remove asserts
ArthurZucker Jul 21, 2022
281fa9e
update code for multiple samples
ArthurZucker Jul 22, 2022
16af74f
Add sample level argument
ArthurZucker Jul 22, 2022
56c0207
no fp16 for now
ArthurZucker Jul 22, 2022
29ec9ab
style
ArthurZucker Jul 22, 2022
0cdf8cb
accelerate support?
ArthurZucker Jul 22, 2022
dc1f57a
update and quality
ArthurZucker Jul 25, 2022
1c2e5a9
delete notebook from wrong folder
ArthurZucker Jul 25, 2022
d90b6f5
Update src/transformers/models/jukebox/modeling_jukebox.py
ArthurZucker Jul 25, 2022
141dbdb
handle return tensor ( no import torch )
ArthurZucker Jul 25, 2022
7c8b228
fix torch not found
ArthurZucker Jul 25, 2022
1798aaf
style
ArthurZucker Jul 25, 2022
7b03a46
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Jul 25, 2022
70f93cd
fixed slow test for 1b lyric
ArthurZucker Jul 25, 2022
a9df0a1
style
ArthurZucker Jul 25, 2022
a857cf1
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Jul 26, 2022
6984553
add VQVAE as pretrained model
ArthurZucker Jul 27, 2022
a4ca921
update indexes
ArthurZucker Jul 27, 2022
7ac9530
update and clean model doc
ArthurZucker Jul 27, 2022
47a220d
update tests
ArthurZucker Jul 27, 2022
fc17643
update test
ArthurZucker Jul 27, 2022
55f2b13
fix slow sampling that is now faster + added all lvl logits
ArthurZucker Jul 27, 2022
f121fce
1b lyrics testing is full and finished
ArthurZucker Jul 27, 2022
7777fc6
style
ArthurZucker Jul 27, 2022
69d9081
update original sampling tests
ArthurZucker Jul 27, 2022
dc42a28
finish tests
ArthurZucker Jul 27, 2022
187dbd2
style
ArthurZucker Jul 27, 2022
322c00d
quality + slow test
ArthurZucker Jul 28, 2022
ba60970
starting code refactoring and renaming
ArthurZucker Jul 28, 2022
e19774a
zs -> music_tokens
ArthurZucker Jul 28, 2022
81e8fbe
update
ArthurZucker Jul 28, 2022
49dbf08
fixe save wav and get_alignment
ArthurZucker Jul 29, 2022
37969d6
update alignment
ArthurZucker Jul 29, 2022
f21c32b
update names and doc
ArthurZucker Jul 29, 2022
9a646cd
Merge branch 'jukebox' of https://github.com/arthurzucker/transformer…
ArthurZucker Jul 29, 2022
cfbfdff
clean resconv1d block : from seq to module list
ArthurZucker Aug 2, 2022
d01fbf8
major VQVAE refactoring everything is modulist + simplified naming
ArthurZucker Aug 5, 2022
7913547
update
ArthurZucker Aug 5, 2022
6dd8d07
clean VQVAE code
ArthurZucker Aug 8, 2022
c097f81
style
ArthurZucker Aug 8, 2022
fc1a2a1
update
ArthurZucker Aug 8, 2022
31af673
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Aug 8, 2022
d388183
major renaming
ArthurZucker Aug 8, 2022
55e83cb
finish weight renaming
ArthurZucker Aug 8, 2022
632258d
isolated remaining variables to rename
ArthurZucker Aug 9, 2022
4f16569
MAJOR UPDATE FULL RENAMING
ArthurZucker Aug 9, 2022
32949c1
style
ArthurZucker Aug 9, 2022
70e9191
major renameing and cleaning parameters from config file
ArthurZucker Aug 10, 2022
44b45c4
cleanup
ArthurZucker Aug 10, 2022
ac29f59
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Aug 10, 2022
83399fe
update conversion and debugged
ArthurZucker Aug 10, 2022
e004808
fix remaining bug, tests pass
ArthurZucker Aug 10, 2022
873cabd
all tests pass (modelling tests)
ArthurZucker Aug 10, 2022
15567f2
should start documenting
ArthurZucker Aug 10, 2022
a271c63
nits
ArthurZucker Aug 10, 2022
dae211d
update
ArthurZucker Aug 11, 2022
57fc92f
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Aug 12, 2022
c90032f
update and support for fp16
ArthurZucker Aug 15, 2022
16fd65d
begin checkpointing res for faster inference and lower memory consump…
ArthurZucker Aug 15, 2022
8b0901b
Merge branch 'jukebox' of https://github.com/arthurzucker/transformer…
ArthurZucker Aug 15, 2022
c3b6e5c
update
ArthurZucker Aug 15, 2022
bd31d24
clean and remove checkpointing
ArthurZucker Aug 17, 2022
042ee40
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Aug 17, 2022
00c614c
clean renamed comments
ArthurZucker Aug 17, 2022
fa622e9
style
ArthurZucker Aug 17, 2022
d55a516
rename prime -> lyric_enc
ArthurZucker Aug 18, 2022
30a7b0d
style
ArthurZucker Aug 18, 2022
e1f7376
Start docstring
ArthurZucker Aug 18, 2022
6c982c0
style
ArthurZucker Aug 18, 2022
f87de9b
update doc
ArthurZucker Aug 18, 2022
98f830a
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Aug 18, 2022
e70b4c8
update
ArthurZucker Aug 18, 2022
a6462e2
fix test update doc
ArthurZucker Aug 19, 2022
ad681ca
style
ArthurZucker Aug 19, 2022
f3590da
update
ArthurZucker Aug 20, 2022
ab624d9
nits
ArthurZucker Sep 12, 2022
9a338ad
Merge branch 'main' into jukebox
ArthurZucker Sep 14, 2022
2d38d3b
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Sep 19, 2022
130260f
update code add docstring remove oneline function
ArthurZucker Sep 19, 2022
7b16748
Merge branch 'main' into jukebox
ArthurZucker Sep 19, 2022
004e78b
modre doc
ArthurZucker Sep 19, 2022
dc626e5
update template doc
ArthurZucker Sep 19, 2022
2eadac7
fix doc
ArthurZucker Sep 19, 2022
374b167
test documentation rendering
ArthurZucker Sep 19, 2022
b96c50d
update doc and config doc
ArthurZucker Sep 19, 2022
b386353
remove cehck cache
ArthurZucker Sep 19, 2022
7c321d8
update tokenization doc and remove asserts
ArthurZucker Sep 19, 2022
b93b078
update test
ArthurZucker Sep 19, 2022
49b2375
improve config doc, remove unused args in vqvae, fix torch issue
ArthurZucker Sep 20, 2022
401b970
nits on tokenizer doc
ArthurZucker Sep 20, 2022
815790b
nits doc builder
ArthurZucker Sep 20, 2022
4e10b01
fix example
ArthurZucker Sep 20, 2022
a6372ce
update modeling doc
ArthurZucker Sep 20, 2022
a17cdf7
update doc
ArthurZucker Sep 20, 2022
a0d51f1
Nit
ArthurZucker Sep 20, 2022
677b27f
remove todos
ArthurZucker Sep 20, 2022
ebad2d1
Apply suggestions from code review
ArthurZucker Sep 21, 2022
e1b086b
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Sep 21, 2022
d2f6eed
Update docs/source/en/model_doc/jukebox.mdx
ArthurZucker Sep 21, 2022
23feb31
Update src/transformers/models/jukebox/__init__.py
ArthurZucker Sep 21, 2022
ea2c6f3
update based on review
ArthurZucker Sep 21, 2022
36c3704
partial update
ArthurZucker Sep 21, 2022
65c067f
update
ArthurZucker Sep 21, 2022
74dbec2
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Oct 3, 2022
d16da73
comment out empyt_cache
ArthurZucker Oct 3, 2022
b6b5c0f
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Oct 18, 2022
4e7468b
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Oct 18, 2022
3e745ee
update based on review
ArthurZucker Oct 18, 2022
273f125
update tqdm usage
ArthurZucker Oct 19, 2022
a67eedc
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Oct 19, 2022
3ac4ed9
fixup
ArthurZucker Oct 19, 2022
13119f3
Apply suggestions from code review
ArthurZucker Oct 21, 2022
6bada42
Update code based on review
ArthurZucker Oct 21, 2022
a8c46b3
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Oct 21, 2022
4307ee6
add fp16 support and test
ArthurZucker Oct 21, 2022
6c22912
clean fp16 test
ArthurZucker Oct 21, 2022
34948de
more cleaning
ArthurZucker Oct 21, 2022
067baab
update doc
ArthurZucker Oct 21, 2022
da6e27e
update doctests
ArthurZucker Oct 21, 2022
2b76a6f
quality
ArthurZucker Oct 24, 2022
693e2c1
fix returns in doc
ArthurZucker Oct 24, 2022
27bfc84
Apply suggestions from code review
ArthurZucker Oct 26, 2022
ef3bd92
Update README.md
ArthurZucker Oct 26, 2022
fa2556c
HUGE refactoring of the code
ArthurZucker Nov 2, 2022
f629b09
fixup
ArthurZucker Nov 2, 2022
28422e7
local test now follow the patterm
ArthurZucker Nov 2, 2022
783b177
merge
ArthurZucker Nov 2, 2022
f739f95
fix last test and correct initialisation pattern
ArthurZucker Nov 3, 2022
15da9b5
fix slopw tests and tokenizer order
ArthurZucker Nov 3, 2022
289f99f
update readmes
ArthurZucker Nov 3, 2022
1ea69d0
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Nov 3, 2022
3584b21
fixcopies and fixup
ArthurZucker Nov 3, 2022
72e128a
fixup
ArthurZucker Nov 3, 2022
9443038
update tips in the readme
ArthurZucker Nov 3, 2022
c55a843
update code
ArthurZucker Nov 4, 2022
d4917c9
renaming here and there + start fixing last tests
ArthurZucker Nov 7, 2022
53e300c
style
ArthurZucker Nov 7, 2022
61d1bb2
nits
ArthurZucker Nov 7, 2022
142cd71
add prior doc
ArthurZucker Nov 7, 2022
099b244
add vqvae docstring
ArthurZucker Nov 7, 2022
5c5bce8
add prior to init and models
ArthurZucker Nov 7, 2022
86ba8f5
update JukeboxConfig
ArthurZucker Nov 7, 2022
43b71b8
format
ArthurZucker Nov 7, 2022
b17f841
JukeboxPrior is tested
ArthurZucker Nov 7, 2022
1fbb470
document JukeboxPrior, ignore non auto configured
ArthurZucker Nov 7, 2022
8f30e8d
cleaning up
ArthurZucker Nov 7, 2022
7ea6fad
nit
ArthurZucker Nov 7, 2022
b4c8649
fix doc build
ArthurZucker Nov 7, 2022
15121b2
clean tests
ArthurZucker Nov 7, 2022
c5f12cd
pretty TQDM : leave = False everywhere
ArthurZucker Nov 7, 2022
52209bf
style
ArthurZucker Nov 7, 2022
897b217
update tests
ArthurZucker Nov 8, 2022
af876a7
add missing documentation
ArthurZucker Nov 8, 2022
0280041
Apply suggestions from code review
ArthurZucker Nov 9, 2022
9d4baab
update doc and variable names, add encoder config in config
ArthurZucker Nov 9, 2022
a5367fd
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Nov 9, 2022
dbb0329
Merge branch 'jukebox' of https://github.com/ArthurZucker/transformer…
ArthurZucker Nov 9, 2022
19f49b6
fixup
ArthurZucker Nov 9, 2022
996a3bb
update
ArthurZucker Nov 9, 2022
d45fa0e
spell check dox
ArthurZucker Nov 9, 2022
ce653b3
fix doc and base model prefixes
ArthurZucker Nov 9, 2022
5b5772c
rename depth to num_layers
ArthurZucker Nov 9, 2022
0ea6ba4
update doc
ArthurZucker Nov 9, 2022
479047b
Update init and add JukeboxPrior and JukeboxVQVAE configs to doc
ArthurZucker Nov 9, 2022
18300a7
fix typechecking
ArthurZucker Nov 9, 2022
84c2ee0
fixup
ArthurZucker Nov 9, 2022
4c98128
update to mask missing keys from other priors
ArthurZucker Nov 9, 2022
7ee9b0d
update initialization
ArthurZucker Nov 9, 2022
9adc384
fix last test
ArthurZucker Nov 10, 2022
daa4cd4
remove super init and clean
ArthurZucker Nov 10, 2022
e636360
fix doctest
ArthurZucker Nov 10, 2022
2a076fa
nits
ArthurZucker Nov 10, 2022
415206c
last fixup
ArthurZucker Nov 10, 2022
d2a5261
code reviews
ArthurZucker Nov 10, 2022
e492cab
remove lambda in ATTENTION_PATTERNS
ArthurZucker Nov 10, 2022
9cbd462
update attention pattern!!
ArthurZucker Nov 10, 2022
e6ef535
tokenizer before model
ArthurZucker Nov 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -320,6 +320,7 @@ Current number of checkpoints: ![](https://img.shields.io/endpoint?url=https://h
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down
1 change: 1 addition & 0 deletions README_es.md
Expand Up @@ -320,6 +320,7 @@ Número actual de puntos de control: ![](https://img.shields.io/endpoint?url=htt
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Expand Up @@ -355,6 +355,7 @@ Flax、PyTorch、TensorFlowをcondaでインストールする方法は、それ
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Expand Up @@ -270,6 +270,7 @@ Flax, PyTorch, TensorFlow 설치 페이지에서 이들을 conda로 설치하는
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down
1 change: 1 addition & 0 deletions README_zh-hans.md
Expand Up @@ -294,6 +294,7 @@ conda install -c huggingface transformers
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (来自 Facebook) 伴随论文 [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) 由 Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed 发布。
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (来自 Berkeley) 伴随论文 [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) 由 Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer 发布。
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (来自 OpenAI) 伴随论文 [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) 由 Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever 发布。
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (来自 Microsoft Research Asia) 伴随论文 [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) 由 Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou 发布。
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (来自 Microsoft Research Asia) 伴随论文 [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) 由 Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou 发布。
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (来自 Microsoft Research Asia) 伴随论文 [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) 由 Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei 发布。
Expand Down
1 change: 1 addition & 0 deletions README_zh-hant.md
Expand Up @@ -306,6 +306,7 @@ conda install -c huggingface transformers
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](https://huggingface.co/docs/transformers/main/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down
2 changes: 2 additions & 0 deletions docs/source/en/_toctree.yml
Expand Up @@ -275,6 +275,8 @@
title: HerBERT
- local: model_doc/ibert
title: I-BERT
- local: model_doc/jukebox
title: Jukebox
- local: model_doc/layoutlm
title: LayoutLM
- local: model_doc/led
Expand Down
2 changes: 2 additions & 0 deletions docs/source/en/index.mdx
Expand Up @@ -108,6 +108,7 @@ The documentation is organized into five sections:
1. **[Hubert](model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Jukebox](model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
1. **[LayoutLMv3](model_doc/layoutlmv3)** (from Microsoft Research Asia) released with the paper [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) by Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei.
Expand Down Expand Up @@ -263,6 +264,7 @@ Flax), PyTorch, and/or TensorFlow.
| Hubert | ❌ | ❌ | ✅ | ✅ | ❌ |
| I-BERT | ❌ | ❌ | ✅ | ❌ | ❌ |
| ImageGPT | ❌ | ❌ | ✅ | ❌ | ❌ |
| Jukebox | ✅ | ❌ | ✅ | ❌ | ❌ |
| LayoutLM | ✅ | ✅ | ✅ | ✅ | ❌ |
| LayoutLMv2 | ✅ | ✅ | ✅ | ❌ | ❌ |
| LayoutLMv3 | ✅ | ✅ | ✅ | ✅ | ❌ |
Expand Down