updated dl_cpu(workflows)- For passing CI-Tests #1135

Atharva-Phatak · 2021-03-23T19:22:41Z

Before submitting (checklist)

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contribution guide?
Did you check the code style? catalyst-make-codestyle && catalyst-check-codestyle (pip install -U catalyst-codestyle).
Did you make sure to update the docs? We use Google format for all the methods and classes.
Did you check the docs with make check-docs?
Did you write any new necessary tests?
Did you check that your code passes the unit tests pytest . ?
Did you add your new functionality to the docs?
Did you update the CHANGELOG?

Description

Related Issue

Type of Change

Examples / docs / tutorials / contributors update
Bug fix (non-breaking change which fixes an issue)
Improvement (non-breaking change which improves an existing feature)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

PS

I know, that I could join slack for pull request discussion.

Note

Mentioned in comment of #1131 . so that CI test will run properly

Scitator · 2021-03-24T05:39:04Z

🤔 could you please try to update pandas requirement to 1.0.0? the time has come 😂
requirements/requiremets-ml.txt file

Scitator · 2021-03-24T05:39:51Z

we should find at least one extra minimal test to work with :)

Atharva-Phatak · 2021-03-24T06:27:30Z

🤔 could you please try to update pandas requirement to 1.0.0? the time has come 😂
requirements/requiremets-ml.txt file

I agree. I wil make the necessary updates mentioned.

While running tests I observed this, we run tests for ddp , gans and segmentation as well. On my laptop(single GPU) ddp failed , which was easy to debug, but wouldnt it be easier if run module specific tests while development and only test them.
When created PR , other tests for ddp will run(which they do).

I think this will make development and adding new modules easier. Just an opinion. Will love to hear more. 🤔

Scitator · 2021-03-24T06:51:53Z

Not sure if I understand you correctly, cause:

we have a bunch of DDP test under our engines folder here https://github.com/catalyst-team/catalyst/tree/master/catalyst/engines/tests
there are also options to skip them if there are no GPUs available
you always could use pytest catalyst/engines (or any other folder) to reduce the number of test cases you are interested in now
during the CI, it's much better to run all the test cases :)

Atharva-Phatak · 2021-03-24T08:10:48Z

@Scitator updated requirments.txt

Scitator · 2021-03-24T20:40:11Z

Extra sys included, we will find the correct way for minimal requirements tests 😂

Atharva-Phatak · 2021-03-25T06:06:56Z

Extra sys included, we will find the correct way for minimal requirements tests joy

yes we will 😂

Scitator · 2021-03-25T06:15:26Z

At least we have dl-cpu (ubuntu-20.04, 3.6, minimal) working already ;)

Atharva-Phatak · 2021-03-25T06:18:44Z

At least we have dl-cpu (ubuntu-20.04, 3.6, minimal) working already ;)

Yes I am happy. We will soon have other versions working.

Scitator · 2021-03-25T06:23:38Z

Honestly speaking, not sure about that – GitHub has an interesting freezing bug here.
Could you please exclude:

- dl-cpu (ubuntu-18.04, 3.6, minimal)
- dl-cpu (ubuntu-18.04, 3.7, minimal)
- dl-cpu (ubuntu-18.04, 3.8, minimal)
- dl-cpu (ubuntu-20.04, 3.7, minimal)
- dl-cpu (ubuntu-20.04, 3.8, minimal)
- dl-cpu (macOS-10.15, 3.7, minimal)

and leave only dl-cpu (ubuntu-20.04, 3.6, minimal)?

All our tests should work under 15mins, so 40+ mins runs are just CI freezing errors :(

Scitator · 2021-03-25T07:05:38Z

Sorry, but you have to keep the matrix and use exclude: in this case ;)

Scitator · 2021-03-25T07:06:05Z

We still need all other test in all other scenarios

Atharva-Phatak · 2021-03-25T07:08:07Z

We still need all other test in all other scenarios

Okay I will correct the mistakes.

Scitator · 2021-03-26T05:57:58Z

could you please return all the check, but exclude only

- dl-cpu (ubuntu-18.04, 3.6, minimal)
- dl-cpu (ubuntu-18.04, 3.7, minimal)
- dl-cpu (ubuntu-18.04, 3.8, minimal)
- dl-cpu (ubuntu-20.04, 3.7, minimal)
- dl-cpu (ubuntu-20.04, 3.8, minimal)
- dl-cpu (macOS-10.15, 3.7, minimal)

?
That requires you to write a precise exclude filter for 3 fields, rather than only 2 right now.

Atharva-Phatak · 2021-03-26T06:16:02Z

@Scitator I have made the changes, please check, if there are any please let me know

* updated dl_cpu * Updated-requirements-ml.txt * corrected the mistake * Update dl_cpu.yml * Update requirements-ml.txt * Kept dl_cpu 20.04-3.6 minimal as requested * Corrected the mistakes * Add requirements parameter * Update dl_cpu.yml * Update dl_cpu.yml Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>

* add movielens * First hitrate metric version * metrge with upstream * fix topk_map * fix error in map * tests for map * top_k ndcg * edit changelog * edit the tabs * check codestyle * check the intent * check the intent * remove trailing whitespace * fixed hitrate * fixed the docs * hitrate * Additive margin softmax (#1131) * Added Additive-margin softmax * update changelog.md * Readme update2 (#1142) * readme * batch overfit fix for reproducibility * batch overfit fix for reproducibility * readme * readme update * extra Config/Hydra API fixes * codestyle * updated dl_cpu(workflows)- For passing CI-Tests (#1135) * updated dl_cpu * Updated-requirements-ml.txt * corrected the mistake * Update dl_cpu.yml * Update requirements-ml.txt * Kept dl_cpu 20.04-3.6 minimal as requested * Corrected the mistakes * Add requirements parameter * Update dl_cpu.yml * Update dl_cpu.yml Co-authored-by: Sergey Kolesnikov <scitator@gmail.com> * fix: `_key_value` for schedulers in case of multiple optimizers fixed (#1146) * Github CI fixes (#1143) * verbose * Utils callbacks (#1127) * quantization.py * onnx and quantization * tracing.py * docs * fix * fix * quantization test * fix test * fix comments * fix comments * Update catalyst/callbacks/__init__.py Co-authored-by: Sergey Kolesnikov <scitator@gmail.com> * tests * tests * tests * tests * tests * pycharm tricks * extra tests * Utils callbacks (#1145) * quantization.py * onnx and quantization * tracing.py * docs * fix * fix * quantization test * fix test * fix comments * fix comments * quantization example * examples * CHANGELOG.md * fix * fix * Apply suggestions from code review Co-authored-by: Sergey Kolesnikov <scitator@gmail.com> * extra tests 2 * extra tests 2 * extra tests 2 * extra tests 2 * extra tests 2 * extra tests 2 * extra tests 2 * extra tests 2 Co-authored-by: Nikita Balagansky <37884009+elephantmipt@users.noreply.github.com> * Engine docs (#1141) * typings & few docs * typing fix; disabled `dist.barrier()` in optimizer step for ddp * docs * docs: fixed long lines with docs * docs fixes * optimizer args * removed empty line Co-authored-by: Dmytro Doroshenko <dimdoroshenko@gmail.com> * v21.03.1 * v21.03: minimal version fix (#1147) * minimal version fix * docs * fix: nested dicts in loaders_params/samplers_params overriding (#1150) * docs (#1152) * Github CI fix (#1148) * fix * fix * Niftireader (#1151) * changes for NiftiReader * adding import statement that was needed and fixing codestyle * updated CHANGELOG * removing unnecesary documentation * fixing codestyle * updated docs and requirements * adding a space * not importing nibabel for docs * fixing typo * adding step for contrib pip caching * fixed codestyle and added a test for reader that uncovered a mistake :) * hopefully fixing imports * adding sergey suggested fix * fixing deploy_push.yml * updating workflows Co-authored-by: Kevin Wang <kevin.wang@affectiva.com> Co-authored-by: Kevin Wang <kcwang26@gsu.edu> * add changelog * solved runtime error * hitrate calculation Co-authored-by: Daniel Chepenko <dchepenk@yahoo-corp.co> Co-authored-by: denyhoof <kde97@yandex.ru> Co-authored-by: Даниил <zkid18@MacBook-Pro.Dlink> Co-authored-by: Atharva Phatak <athp456@gmail.com> Co-authored-by: Sergey Kolesnikov <scitator@gmail.com> Co-authored-by: Yauheni Kachan <19803638+bagxi@users.noreply.github.com> Co-authored-by: Nikita Balagansky <37884009+elephantmipt@users.noreply.github.com> Co-authored-by: Dmytro Doroshenko <dimdoroshenko@gmail.com> Co-authored-by: Kevin Wang <kcwang@ymail.com> Co-authored-by: Kevin Wang <kevin.wang@affectiva.com> Co-authored-by: Kevin Wang <kcwang26@gsu.edu>

* updated dl_cpu * Updated-requirements-ml.txt * corrected the mistake * Update dl_cpu.yml * Update requirements-ml.txt * Kept dl_cpu 20.04-3.6 minimal as requested * Corrected the mistakes * Add requirements parameter * Update dl_cpu.yml * Update dl_cpu.yml Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>

updated dl_cpu

842e1a2

Atharva-Phatak requested review from bagxi, ditwoo and Scitator as code owners March 23, 2021 19:22

Atharva-Phatak changed the title ~~updated dl_cpu- For passing CI-Tests~~ updated dl_cpu(workflows)- For passing CI-Tests Mar 23, 2021

Updated-requirements-ml.txt

253ed25

Atharva-Phatak and others added 2 commits March 24, 2021 14:21

corrected the mistake

b3aabdd

Update dl_cpu.yml

695e201

Update requirements-ml.txt

e863599

Kept dl_cpu 20.04-3.6 minimal as requested

15e61a9

Corrected the mistakes

5f5028d

Add requirements parameter

3bbf066

Atharva-Phatak and others added 3 commits March 27, 2021 21:58

Merge branch 'master' into master

6a3322e

Update dl_cpu.yml

5121a3b

Update dl_cpu.yml

992ac9d

Merge branch 'master' into master

2108cbb

Scitator merged commit 45fe1bd into catalyst-team:master Mar 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updated dl_cpu(workflows)- For passing CI-Tests #1135

updated dl_cpu(workflows)- For passing CI-Tests #1135

Atharva-Phatak commented Mar 23, 2021

Scitator commented Mar 24, 2021 •

edited

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 24, 2021

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 24, 2021

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 25, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 25, 2021

Scitator commented Mar 25, 2021

Scitator commented Mar 25, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 26, 2021 •

edited

Atharva-Phatak commented Mar 26, 2021

updated dl_cpu(workflows)- For passing CI-Tests #1135

updated dl_cpu(workflows)- For passing CI-Tests #1135

Conversation

Atharva-Phatak commented Mar 23, 2021

Before submitting (checklist)

Description

Related Issue

Type of Change

PR review

Note

Scitator commented Mar 24, 2021 • edited

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 24, 2021

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 24, 2021

Scitator commented Mar 24, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 25, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 25, 2021

Scitator commented Mar 25, 2021

Scitator commented Mar 25, 2021

Atharva-Phatak commented Mar 25, 2021

Scitator commented Mar 26, 2021 • edited

Atharva-Phatak commented Mar 26, 2021

Scitator commented Mar 24, 2021 •

edited

Scitator commented Mar 26, 2021 •

edited