-
Notifications
You must be signed in to change notification settings - Fork 385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
api: Add params_show
#3459
Merged
Merged
api: Add params_show
#3459
Changes from 29 commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
83fd85d
api: Add params_show.
daavoo 59e38bc
Updates
daavoo 044cade
Apply suggestions from code review
jorgeorpinel 6d16c25
sidebar: Use alphabetical order for api-reference
daavoo 9bbbe25
Applied suggestions
daavoo 3d33521
Motivation for remote `repo`
daavoo 1f7ca1f
Merge branch 'master' into api-get-params
jorgeorpinel cc11224
Update content/docs/api-reference/params_show.md
jorgeorpinel 6490cc9
Merge branch 'api-get-params' of github.com:iterative/dvc.org into ap…
jorgeorpinel c3b5e3b
ref: params_show *targets desc update
jorgeorpinel 6b33374
ref: clarify params_show() descp
jorgeorpinel a631f32
ref: remove simple example from params_show() and
jorgeorpinel e19dda0
ref: simplify params)show() sample
jorgeorpinel c2d9ec8
ref: simplify params_show(rev)
jorgeorpinel c72d752
ref: improve params_show(deps)
jorgeorpinel cb77cb4
ref: improve params_show(stages)
jorgeorpinel 92da7c5
ref: improve project version example of params_show()
jorgeorpinel 23d7099
ref: improve params_show(targets) example and
jorgeorpinel 090feaa
ref: reorg params_show() examples per
jorgeorpinel 6e3b8a7
ref: shorten params_show(repo) example
jorgeorpinel 834e47b
Merge branch 'master' into api-get-params
jorgeorpinel 3060eef
api: json->py dict
jorgeorpinel 7529503
Apply suggestions from code review
jorgeorpinel 9572e72
api: move params_show(stages) example up, and
jorgeorpinel 5032c5a
Merge branch 'api-get-params' of github.com:iterative/dvc.org into ap…
jorgeorpinel 3ced721
Merge branch 'master' into api-get-params
jorgeorpinel 1af16ab
ref: keep params_show examples hypothetical
jorgeorpinel 2e02967
ref: params_show( stages before repo
jorgeorpinel 2a70b7d
ref: more realistic params_show(stages) example
jorgeorpinel 590dfdc
Merge branch 'main' into api-get-params
jorgeorpinel 4f755c0
ref: clarify on params_show(*targets)
jorgeorpinel 1c53cc6
Apply suggestions from code review
jorgeorpinel b8cbaf1
Apply suggestions from review
daavoo 712464d
Remove reference to example repo
daavoo 2525cf9
Update example
daavoo 24f18aa
reorder
daavoo 87ebba4
Wording
daavoo e8596d5
Remove example get started reference
daavoo File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,198 @@ | ||
# dvc.api.params_show() | ||
|
||
Load <abbr>parameters</abbr> (name and values) tracked in a <abbr>DVC | ||
project</abbr>. | ||
|
||
```py | ||
def params_show( | ||
*targets: str, # Optional | ||
stages: Optional[Union[str, Iterable[str]]] = None, | ||
repo: Optional[str] = None, | ||
rev: Optional[str] = None, | ||
deps: bool = False, | ||
daavoo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) -> Dict: | ||
``` | ||
|
||
## Usage: | ||
|
||
```py | ||
import dvc.api | ||
|
||
params = dvc.api.params_show() | ||
``` | ||
|
||
## Description | ||
|
||
Retrieves <abbr>params</abbr> keys and values from a <abbr>DVC project</abbr> | ||
and returns a dictionary, such as: | ||
|
||
```json | ||
{ | ||
"split": 0.2, | ||
"seed": 20170428 | ||
} | ||
``` | ||
|
||
Without arguments, this function will retrieve all params from all tracked | ||
parameter files (used in any `dvc.yaml` file). This applies to the current | ||
dberenbaum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
project version when using Git (including any changes in the working tree). | ||
|
||
The function parameters (below) let you restrict what's retrieved. | ||
|
||
## Parameters | ||
|
||
- `*targets` - paths to one or more valid parameter file(s) to retrieve params | ||
from. For example, `"params.py, myparams.toml"`. If no `targets` are provided, | ||
dberenbaum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
all param files tracked in any `dvc.yaml` will be used by default. Note that | ||
explicit targets don't have to be used in a `dvc.yaml` (unless `deps=True`). | ||
|
||
- `repo` - specifies the location of the DVC project. It can be a URL or a file | ||
system path. Both HTTP and SSH protocols are supported for online Git repos | ||
(e.g. `[user@]server:project.git`). _Default_: The current project (found by | ||
walking up from the current working directory tree). | ||
|
||
- `stages` - one or more names of the stage(s) to retrieve params from. | ||
_Default_: `None` (all parameters from all stages will be retrieved). | ||
|
||
- `rev` - Git commit (any [revision](https://git-scm.com/docs/revisions) such as | ||
a branch or tag name, a commit hash or an | ||
[experiment](/doc/command-reference/exp) name). If `repo` is not a Git repo, | ||
this option is ignored. _Default_: `None` (current working tree will be used) | ||
jorgeorpinel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- `deps` - whether to retrieve only params that are stage dependencies. Accepts | ||
`True` or `False` (_default_). | ||
|
||
## Example: Filter by stage name(s) | ||
|
||
> Working on https://github.com/iterative/example-get-started, file | ||
> `src/featurization.py`. | ||
daavoo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
`stages` can be a single name (string): | ||
|
||
```git | ||
+import json | ||
-import yaml | ||
|
||
-params = yaml.safe_load(open("params.yaml"))["featurize"] | ||
+import dvc.api | ||
+ | ||
+params = dvc.api.get_params(stages="featurize") | ||
jorgeorpinel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+params = json.dumps(params, indent=2) | ||
jorgeorpinel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
... | ||
-max_features = params["max_features"] | ||
-ngrams = params["ngrams"] | ||
+max_features = params["featurize"]["max_features"] | ||
+ngrams = params["featurize"]["ngrams"] | ||
``` | ||
|
||
Or an iterable of strings: | ||
|
||
```py | ||
import json | ||
import dvc.api | ||
params = dvc.api.get_params(stages=["featurize", "train"]) | ||
jorgeorpinel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
```json | ||
{ | ||
"featurize": { | ||
"max_features": 200, | ||
"ngrams": 2 | ||
}, | ||
"train": { | ||
"seed": 20170428, | ||
"n_est": 50, | ||
"min_split": 0.01 | ||
} | ||
} | ||
``` | ||
|
||
## Example: Load specific parameter file(s) | ||
|
||
You can pass any valid param file path as target to load all of the parameters | ||
defined in it: | ||
|
||
```py | ||
import json | ||
import dvc.api | ||
params = dvc.api.params_show("params.yaml") | ||
``` | ||
|
||
```json | ||
{ | ||
"run_mode": "prod", | ||
"configs": { | ||
"dev": "configs/params_dev.yaml", | ||
... | ||
``` | ||
|
||
Or multiple path targets: | ||
|
||
```py | ||
import json | ||
import dvc.api | ||
params = dvc.api.params_show( | ||
"configs/params_dev.yaml", "configs/params_prod.yaml") | ||
``` | ||
|
||
```json | ||
{ | ||
"configs/params_prod.yaml:run_mode": "prod", | ||
"configs/params_prod.yaml:config_file": "configs/params_prod.yaml", | ||
... | ||
"configs/params_dev.yaml:run_mode": "dev", | ||
"configs/params_dev.yaml:config_file": "configs/params_dev.yaml", | ||
... | ||
``` | ||
|
||
## Example: Use a remote DVC repository | ||
|
||
You can use the `repo` argument to retrieve parameters from any <abbr>DVC | ||
repository</abbr> without having to clone it locally. | ||
|
||
```py | ||
import json | ||
import dvc.api | ||
params = dvc.api.get_params( | ||
repo="https://github.com/iterative/demo-fashion-mnist") | ||
daavoo marked this conversation as resolved.
Show resolved
Hide resolved
daavoo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
```json | ||
{ | ||
"train": { | ||
"batch_size": 128, | ||
"hidden_units": 64, | ||
"dropout": 0.4, | ||
... | ||
``` | ||
|
||
## Example: Specify a project version | ||
|
||
> Working on https://github.com/iterative/example-get-started | ||
|
||
You can retrieve params from arbitrary Git commits, for example a branch name: | ||
|
||
```py | ||
import json | ||
import dvc.api | ||
params = dvc.api.get_params(rev="tune-hyperparams") | ||
jorgeorpinel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
```json | ||
{ | ||
"prepare": { | ||
"split": 0.2, | ||
"seed": 20170428 | ||
}, | ||
"featurize": { | ||
"max_features": 200, | ||
"ngrams": 2 | ||
}, | ||
"train": { | ||
"seed": 20170428, | ||
"n_est": 100, | ||
"min_split": 8 | ||
} | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be moved above
repo
?stages
seems more important, and it makes sense to me to put it right aftertargets
and keeprepo
andrev
together.This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the signature from what I can tell. I don't think we should alter it for docs (other than that one comment I added).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said if you plan to change the signature in the core repo then sure. Lmk
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that those are all keyword arguments so the order doesn't really affect the behavior. We could also update the order in core, if it makes more sense
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't mind updating in core, but to me it's more important to update here to start with more common use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. I moved
stages
one position up. Why not all the way up then? Since we deem the stage-filtering example most important... Which goes back to iterative/dvc#7613 (review)I think that's ideal because at some point this may get copy-pasted again from there (on some update) and our custom docs order is gone.