Skip to content

Issues: stanford-crfm/helm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

209 Open 819 Closed
209 Open 819 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Mini-leaderboards show worst models first when worst_is_better: true
#2711 opened Jun 4, 2024 by yifanmai
Annotator issues
#2697 opened May 31, 2024 by yifanmai
4 tasks
Upgrade transformers~=4.41 dependencies Pull requests that update a dependency file
#2695 opened May 31, 2024 by teetone
Potential Inclusion of the Phi-3 model family
#2692 opened May 30, 2024 by bryanzhou008
Support for Hugging Face Local Models
#2690 opened May 30, 2024 by NoushNabi
1
warning when on outdated site frontend Frontend issues
#2681 opened May 24, 2024 by farzaank
@farzaank
Unable to download HELM leaderboard results (v1.3.0)
#2680 opened May 24, 2024 by jasonwright38
6
Change gsutil rsync to gcloud storage rsync in documentation documentation Improvements or additions to documentation
#2676 opened May 23, 2024 by yifanmai
Make leaderboard cards view better on mobile frontend Frontend issues
#2662 opened May 20, 2024 by farzaank
@farzaank
Don't set trust_remote_code=True by default in HuggingFaceClient
#2645 opened May 13, 2024 by yifanmai
HuggingFaceClient / HuggingFaceTokenizer with use_fast=False and allow_remote_code=True
#2639 opened May 10, 2024 by yifanmai
1
max_train_instances should accept integer arguments
#2638 opened May 10, 2024 by yifanmai
Duplication of fields across *_schema.yaml files
#2631 opened May 8, 2024 by yifanmai
1 of 5 tasks
Update schema and frontend copy for HELM Medical
#2630 opened May 8, 2024 by yifanmai
@farzaank
CRFM logo doesn't take you anywhere in the front-end header
#2625 opened May 8, 2024 by farzaank
@farzaank
Fix instance level metrics being way too far to the right
#2619 opened May 7, 2024 by farzaank
@farzaank
Unable to run evaluation for med_QA dataset
#2598 opened May 2, 2024 by richardzhuang0412
8
Missing Dependency chardet
#2573 opened Apr 16, 2024 by danielz02
2
Please include gpt-4-0125-preview
#2560 opened Apr 10, 2024 by jamesmalin
1
helm-summarize gives extra weight to runs in multiple groups framework methodology Evaluation methodology p2 Priority 2 (Good to have for release)
#2541 opened Apr 2, 2024 by yifanmai
1
Add validation for schema YAML files good first issue Good for newcomers
#2528 opened Mar 29, 2024 by yifanmai
8 tasks
Allow specifying MongoDB password via alternate means good first issue Good for newcomers
#2522 opened Mar 28, 2024 by yifanmai
Bold or delineate subsections in Predictions pages under Request details frontend Frontend issues
#2513 opened Mar 26, 2024 by farzaank
Problems and questions on the reproducibility of experiments user question
#2504 opened Mar 25, 2024 by CristianCosci
4
Requests using messages do not count against quota on crfm-models
#2502 opened Mar 22, 2024 by yifanmai
ProTip! Adding no:label will show everything without a label.