Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add table of issue type info and relevant column name descriptions #1100

Merged
merged 23 commits into from
Jun 6, 2024

Conversation

elisno
Copy link
Member

@elisno elisno commented Apr 15, 2024

Addresses #1081

This PR improves the Datalab Issue Types guide.

It adds a table for the different tasks that Datalab supports. It lists out the

  • Names of the issues it can find.
  • Whether it is searched by default (otherwise it's accessible via the issue_types argument in Datalab.find_issues().
  • A collection of relevant columns in the Datalab.issues dataframe.
  • A column that lists out what types of inputs to Datalab.find_issues() are required to successfully run the issue check.

Here's a screenshot of the table (still WIP):
image

@elisno elisno marked this pull request as draft April 15, 2024 23:28
@elisno elisno marked this pull request as ready for review April 17, 2024 18:54
@@ -56,6 +59,80 @@ To handle mislabeled examples, you can either filter out the data with label iss

Learn more about the method used to detect label issues in our paper: `Confident Learning: Estimating Uncertainty in Dataset Labels <https://arxiv.org/abs/1911.00068>`_

.. testsetup:: *
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This testsetup block will be executed for all doctests blocks (..testcode), they just won't run the doctest until we set it up in CI.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These testsetup cells are not visible in the docs.

@@ -46,6 +46,7 @@
"sphinx.ext.napoleon",
"nbsphinx",
"sphinx.ext.autodoc",
"sphinx.ext.autosectionlabel",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is added to allow us to link to local section headings.

Copy link

codecov bot commented Jun 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.03%. Comparing base (0221f8b) to head (c396283).
Report is 28 commits behind head on master.

Current head c396283 differs from pull request most recent head 11dc988

Please upload reports for the commit 11dc988 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1100      +/-   ##
==========================================
- Coverage   96.18%   96.03%   -0.15%     
==========================================
  Files          76       80       +4     
  Lines        5996     6074      +78     
  Branches      992     1003      +11     
==========================================
+ Hits         5767     5833      +66     
- Misses        135      142       +7     
- Partials       94       99       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@jwmueller jwmueller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Pushed many changes, so please review my commits closely before you merge this

@@ -0,0 +1,186 @@
.. tabs::

.. tab:: Classification task
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened another TODO related to this table:
#1133

But can save for the future

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree

@elisno elisno merged commit d9f589e into cleanlab:master Jun 6, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants