New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Select 4 or 5 datasets #7628

Open

4 tasks done

Tracked by

mrm1001 opened this issue May 2, 2024 · 0 comments

Open

4 tasks done

Tracked by

Select 4 or 5 datasets #7628

mrm1001 opened this issue May 2, 2024 · 0 comments

Assignees

Labels

P1

Member

mrm1001 commented May 2, 2024 •

edited by davidsbatista

Select datasets by looking at Notion page
The datasets need to have the following properties:
- at least one should be financial or legal and raw data needs to be in structured pdfs
- least one should be about support/help centre
- there should be one that has been used in other benchmarks (maybe based on wikipedia)
- they should all have a set of labels so that we can get performance metrics from them

mrm1001 mentioned this issue

Benchmark existing techniques using evaluation harness #7625

Open

masci added the P2 label

mrm1001 added P1 and removed P2 labels

masci assigned davidsbatista

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment