Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add oversampling strategy iterable datasets interleave #5036

Merged
merged 12 commits into from
Sep 30, 2022
Merged

Add oversampling strategy iterable datasets interleave #5036

merged 12 commits into from
Sep 30, 2022

Commits on Aug 25, 2022

  1. add IterableDatasets oversampling strategy in interleave_datasets

    Yoach Lacombe committed Aug 25, 2022
    Configuration menu
    Copy the full SHA
    84769db View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2022

  1. add oversampling strategy for iterable datasets

    @ylacombe
    ylacombe committed Sep 25, 2022
    Configuration menu
    Copy the full SHA
    bfb2f1d View commit details
    Browse the repository at this point in the history

  2. update docs regarding interleave_datasets with oversampling strategy … 

    …for IterableDatasets
    @ylacombe
    ylacombe committed Sep 25, 2022
    Configuration menu
    Copy the full SHA
    2467ba9 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2022

  1. correct the is_exhausted index in order to avoid unconsistent behavior

    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    026ca36 View commit details
    Browse the repository at this point in the history

  2. refactoring of RandomlyCyclingMultiSourcesExamplesIterable and Cyclin… 

    …gMultiSourcesExamplesIterable to avoid code redundancy
    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    d303368 View commit details
    Browse the repository at this point in the history

  3. add hardcoded test for interleaving IterableDatasets with oversamplin… 

    …g strategy
    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    f4a65ba View commit details
    Browse the repository at this point in the history

  4. add test for interleaving IterableDatasets with oversampling strategy

    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    16b3b46 View commit details
    Browse the repository at this point in the history

  5. modify tests for cycling multiple IterableDatasets

    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    c58968a View commit details
    Browse the repository at this point in the history

  6. update the interleave_datasets doc

    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    b9f553d View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into add-oversampling-strategy-iterable-datasets-… 

    …interleave
    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    459d8c4 View commit details
    Browse the repository at this point in the history

  8. make style following conflict

    @ylacombe
    ylacombe committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    93132af View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2022

  1. Update src/datasets/iterable_dataset.py 

    Remove resetting of empty iterators

Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>
    @ylacombe @lhoestq
    ylacombe and lhoestq committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    e79809b View commit details
    Browse the repository at this point in the history