Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero-shot accuracy on ImageNet (in the CLIP setting) is lower than the number reported in the paper #6

Open
shyammarjit opened this issue Mar 14, 2024 · 2 comments

Comments

@shyammarjit
Copy link

Zero-shot accuracy on ImageNet (in the CLIP setting)

Top-1 accuracy: 77.15
Top-5 accuracy: 95.51

Paper reported accuracy on ImageNet is (Top-1): 88.5

@209ye
Copy link

209ye commented Mar 20, 2024

The 88.5 accuracy mentioned in the paper here should be the 10B model, which seems to have not been announced yet. The published 0.4b model looks at the data in the paper and is (Top-1) 78.5.

@shyammarjit
Copy link
Author

How do I load the 10B model? Is it open-sourced yet?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants