Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH mini bench #641

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

ENH mini bench #641

wants to merge 4 commits into from

Conversation

tomMoral
Copy link
Member

Fixes #605.

This implements decorator to create mini benchmarks runnable with benchopt run --mini mini_file.py.
This allows to run the following benchmark:

from benchopt.mini import solver, dataset, objective
import jax

@dataset(
    size=100,
    random_state=0
)
def simulated(size, random_state):
    key = jax.random.PRNGKey(random_state)
    key, subkey = jax.random.split(key)
    X = jax.random.normal(key, (size,))
    return dict(X=X)


@solver(
    name="Solver 1",
    lr=[1e-2, 1e-3]
)
def solver1(n_iter, X, lr):
    beta = X
    for i in range(n_iter):
        beta -= lr * beta

    return dict(beta=beta)


@objective(name="Benchmark HVP")
def evaluate(beta):
    return dict(value=(0.5 * beta.dot(beta)).item())

Not completely sure this is useful but got it done by playing with another benchmark, it would be nice to have feedback on whether it seems usable or not.

(No doc or test yet)

Checks before merging PR

  • added documentation for any new feature
  • added unit test
  • edited the what's new (if applicable)

@codecov-commenter
Copy link

Codecov Report

Merging #641 (724f685) into main (9955d4c) will decrease coverage by 1.21%.
The diff coverage is 29.62%.

❗ Current head 724f685 differs from pull request most recent head 51c1de3. Consider uploading reports for the commit 51c1de3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main     #641      +/-   ##
==========================================
- Coverage   56.14%   54.94%   -1.21%     
==========================================
  Files          45       46       +1     
  Lines        2882     3016     +134     
  Branches      530      550      +20     
==========================================
+ Hits         1618     1657      +39     
- Misses       1144     1238      +94     
- Partials      120      121       +1     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement decorators to allow on-the-fly benchmarks
2 participants