Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda wrapper callback #1153

Merged
merged 38 commits into from Apr 15, 2021
Merged

Lambda wrapper callback #1153

merged 38 commits into from Apr 15, 2021

Conversation

elephantmipt
Copy link
Collaborator

Before submitting (checklist)

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contribution guide?
  • Did you check the code style? catalyst-make-codestyle && catalyst-check-codestyle (pip install -U catalyst-codestyle).
  • Did you make sure to update the docs? We use Google format for all the methods and classes.
  • Did you check the docs with make check-docs?
  • Did you write any new necessary tests?
  • Did you check that your code passes the unit tests pytest . ?
  • Did you add your new functionality to the docs?
  • Did you update the CHANGELOG?
  • Did you run colab minimal CI/CD with latest and minimal requirements?

Description

Related Issue

Type of Change

  • Examples / docs / tutorials / contributors update
  • Bug fix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves an existing feature)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

PS

  • I know, that I could join slack for pull request discussion.

catalyst/callbacks/lambda_wrapper.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_wrapper.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_wrapper.py Outdated Show resolved Hide resolved
@pep8speaks
Copy link

pep8speaks commented Apr 1, 2021

Hello @elephantmipt! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-04-15 15:21:04 UTC

catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
@Scitator
Copy link
Member

Scitator commented Apr 1, 2021

btw, could you please also add this callback usage to the test and examples?

@mergify
Copy link

mergify bot commented Apr 2, 2021

This pull request is now in conflicts. @elephantmipt, could you fix it? 🙏

catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/tests/test_recsys.py Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/lambda_preprocess.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Show resolved Hide resolved
Copy link
Member

@Scitator Scitator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we simplify the code a bit?
ley's support only 2 cases:

  1. tuple-in, tuple-out, specified with input_key, output_key, for example:
  • input_key=logits, output_key=scores
  • input_key=["logits", "weights"], output_key=scores
  • input_key=emgeddings, output_key=["pos_embeddings", "neg_embeddings"]

in this case, could simply use:

def batch_handler(self, runner):
    batch_in = [runner.batch[key] for key in self.input_key]
    batch_out = self.handler(batch_in)
    runner.batch.update(**{key:value for key, value in zip(self.output_key, batch_out)})
  1. dict-in, dict-out, specified with input_key=None, output_key=None
    usage:
def batch_handler(self, runner):
    runner.batch = self.handler(runner.batch)

catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
@mergify mergify bot dismissed Scitator’s stale review April 14, 2021 06:05

Pull request has been modified.

README.md Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/tests/test_transform_kornia.py Outdated Show resolved Hide resolved
catalyst/tests/test_recsys.py Outdated Show resolved Hide resolved
elephantmipt and others added 2 commits April 14, 2021 10:09
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
@mergify mergify bot dismissed Scitator’s stale review April 14, 2021 07:09

Pull request has been modified.

elephantmipt and others added 2 commits April 14, 2021 10:12
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
catalyst/callbacks/batch_transform.py Outdated Show resolved Hide resolved
@mergify mergify bot dismissed Scitator’s stale review April 15, 2021 15:02

Pull request has been modified.

@Scitator Scitator merged commit ec9c2a2 into master Apr 15, 2021
@mergify mergify bot deleted the lambda_wrapper_callback branch April 15, 2021 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants