Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exp init: Revisit type="dl" #7534

Closed
Tracked by #225
daavoo opened this issue Apr 1, 2022 · 5 comments · Fixed by #7603
Closed
Tracked by #225

exp init: Revisit type="dl" #7534

daavoo opened this issue Apr 1, 2022 · 5 comments · Fixed by #7603
Assignees
Labels
A: experiments Related to dvc exp

Comments

@daavoo
Copy link
Contributor

daavoo commented Apr 1, 2022

Once iterative/dvclive#225 is solved, the live section will be deprecated in favor of using metrics and plots.

How should exp init behave when type="dl"?

@daavoo daavoo added the A: experiments Related to dvc exp label Apr 1, 2022
@daavoo
Copy link
Contributor Author

daavoo commented Apr 1, 2022

Would it make sense for type="dl" to generate:

stages:
  train:
    cmd: python train.py
    deps:
      - train.py
    metrics:
      - dvclive.json:
          cache: false
    plots:
      - dvclive/scalars:
          cache: false

Where dvclive would be replaced by the value passed to --live (if any) ?

@daavoo
Copy link
Contributor Author

daavoo commented Apr 1, 2022

cc @dberenbaum @skshetry

@daavoo daavoo changed the title exp init: Revisit `type="dl" exp init: Revisit type="dl" Apr 1, 2022
@dberenbaum
Copy link
Contributor

@daavoo Would you be up to contribute this change?

@daavoo
Copy link
Contributor Author

daavoo commented Apr 4, 2022

@daavoo Would you be up to contribute this change?

If you all agree on proposed workaround, yes

@dberenbaum
Copy link
Contributor

Sorry, I think I was too quick to approve this. Looking at it now, it seems more transparent to:

  • Make --live optional and off by default for both default and dl. This makes --type dl seem less useful since it only differs by including checkpoints, but I don't think it's bad that we are unifying and simplifying.
  • Make --live independent from whether to include checkpoints: true.
  • Make --live mutually exclusive with metrics and plots.
  • If --live and --interactive are both present, don't ask for metrics or plots.

This hopefully makes it a simple convenience to replace metrics and plots easily when using dvclive.

Again, sorry for not catching this earlier.

Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: experiments Related to dvc exp
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants