New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add XGLM models #14876
Add XGLM models #14876
Conversation
52fb413
to
2fe52b4
Compare
in social value tasks such as hate speech detection in five languages and find it has limitations similar to comparable sized GPT-3 models.* | ||
|
||
|
||
This model was contributed by [Suraj](https://huggingface.co/valhalla). The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/xglm). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can add some prompt examples here - but happy to do this in a follow-up PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, looks very clean to me already!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for adding this new model!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great, thanks @patil-suraj!
@@ -910,6 +910,35 @@ def converted(self) -> Tokenizer: | |||
return tokenizer | |||
|
|||
|
|||
class XGLMConverter(SpmConverter): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
for model_class_name in self.all_model_classes: | ||
model = model_class_name.from_pretrained("facebook/xglm-564M") | ||
outputs = model(np.ones((1, 1))) | ||
self.assertIsNotNone(outputs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could there be an integration test here as well?
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
What does this PR do?
This PR adds the XGLM model: code, paper