Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(launch): add a new --build flag for building and then pushing the image to a queue #4061

Merged
merged 127 commits into from Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
494d193
updated CLI to support build flag
gtarpenning Aug 1, 2022
4c7d8eb
changed entrypoint defaults and breaking tag string related to github…
gtarpenning Aug 1, 2022
2d66a99
build fails gracefully git proj without requirements
gtarpenning Aug 1, 2022
2b53477
basic implementation of --build working, NOT job
gtarpenning Aug 1, 2022
d6e9ca7
cleanup
gtarpenning Aug 1, 2022
4ca194d
merge from master
gtarpenning Aug 2, 2022
6ecfe3b
major improvements to structure, working job artifact creation
gtarpenning Aug 2, 2022
4e1ab71
temp commit, lock in current entrypoint setup
gtarpenning Aug 3, 2022
b4d2cd7
Now actually adds files to docker image, working job construction
gtarpenning Aug 3, 2022
10cdb2e
still tons of debugging to do, wandb uri pathway not working
gtarpenning Aug 4, 2022
b6a2e68
chaned hardcoding for local dev
gtarpenning Aug 5, 2022
9ff7a2d
Merge branch 'master' of github.com:wandb/client into launch-build-on…
gtarpenning Aug 5, 2022
e708301
refactor changes promoting readability and correct typeswq
gtarpenning Aug 5, 2022
78015cb
remove errant logfile that snuck by .gitignore
gtarpenning Aug 5, 2022
bf604af
removed errant spaces
gtarpenning Aug 5, 2022
602388c
more formatting and cleanup
gtarpenning Aug 5, 2022
5a6c477
Does local-container affect mocked tests?
gtarpenning Aug 5, 2022
f880b76
fix typo
gtarpenning Aug 5, 2022
cba2ef0
major refactor, TODO: testing X(
gtarpenning Aug 10, 2022
f35556d
lint
gtarpenning Aug 10, 2022
2f1367d
first couple tests
gtarpenning Aug 10, 2022
4790054
merge
gtarpenning Aug 12, 2022
2e130ee
temp review edit commit
gtarpenning Aug 12, 2022
4b55253
merge
gtarpenning Aug 12, 2022
7fbefdf
comment from pr review
gtarpenning Aug 12, 2022
d77e29d
more possible testing aparatus for build on queue, not perfect
gtarpenning Aug 12, 2022
1d44905
do we need a CLI test that actually builds on queue?
gtarpenning Aug 16, 2022
eceda21
local merge
gtarpenning Aug 23, 2022
862f093
lint
gtarpenning Aug 24, 2022
d803a35
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Aug 24, 2022
9e76146
Merge branch 'master' of github.com:wandb/client into launch-build-on…
gtarpenning Aug 25, 2022
ce67222
finally a decent test for build!
gtarpenning Aug 26, 2022
7562756
merge with kyle
gtarpenning Aug 26, 2022
afa0acd
im going to invite mypy to my wedding
gtarpenning Aug 26, 2022
8696c33
all hail isort
gtarpenning Aug 26, 2022
0b125ba
small cleanup of variable names and removed old tests
gtarpenning Aug 26, 2022
3a2756e
small linter changes to get around gross mypy typing
gtarpenning Aug 26, 2022
11a0fc9
works locally, why does CircleCI fail?
gtarpenning Aug 26, 2022
70ac345
Merge branch 'master' of github.com:wandb/client into launch-build-on…
gtarpenning Aug 26, 2022
6b82713
circle ci-joe
gtarpenning Aug 26, 2022
4327486
might this appease our circle overlords?
gtarpenning Aug 26, 2022
e2d1593
what does _Error 1451: Cannot delete or update a parent row: a foreig…
gtarpenning Aug 27, 2022
3ed1360
Circle can't delete a user with live runqueues
gtarpenning Aug 27, 2022
f3911e0
testing
gtarpenning Aug 27, 2022
b9ef97a
small changes to setup and testing
gtarpenning Aug 29, 2022
aa74391
okay now for real
gtarpenning Aug 30, 2022
d44399a
lateral progress isn't progress
gtarpenning Aug 30, 2022
0ee9b79
lints
gtarpenning Aug 30, 2022
19fa3b5
testing for build wip
gtarpenning Sep 28, 2022
94ee214
small test hardenings
gtarpenning Sep 28, 2022
8791a8a
lint
gtarpenning Sep 28, 2022
ccf6170
Merge branch 'master' of github.com:wandb/client into launch-build-on…
gtarpenning Sep 28, 2022
625f4a2
using run.init to create a project is such a headache
gtarpenning Sep 28, 2022
c2e0e74
sorting lint
gtarpenning Sep 28, 2022
bbe9a52
test passing!
gtarpenning Sep 28, 2022
341f3d8
isort bane of my existence
gtarpenning Sep 28, 2022
2ee360b
patched verify docker install
gtarpenning Sep 28, 2022
52c25b1
updated test run path to be external
gtarpenning Sep 29, 2022
ca483ba
added more extensive testing and support for configs
gtarpenning Sep 29, 2022
2cbe092
merge
gtarpenning Sep 29, 2022
f2f7474
removed assert that did nothing
gtarpenning Sep 29, 2022
7385367
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 4, 2022
7147152
small review comment fix
gtarpenning Oct 4, 2022
d51ea96
changed job path, updated test
gtarpenning Oct 4, 2022
83e9b55
merge
gtarpenning Oct 6, 2022
461d26a
cli message update
gtarpenning Oct 6, 2022
a2c298b
corrected repository path
gtarpenning Oct 6, 2022
59b97a3
test lint
gtarpenning Oct 6, 2022
5ad9dc4
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 11, 2022
f7833c5
updated tests with new repository flag
gtarpenning Oct 12, 2022
8c68bdf
isort
gtarpenning Oct 13, 2022
62854d6
fixed tiny bug with big implications, thanks Kyle
gtarpenning Oct 13, 2022
8b27d26
removed errant log and added a space
gtarpenning Oct 13, 2022
8b6e078
removed dupe function
gtarpenning Oct 13, 2022
48f32d0
fixed sweep creation with repo
gtarpenning Oct 13, 2022
b32a6e7
removed wrong sweep test path
gtarpenning Oct 13, 2022
14a2ac5
testing apparatus changes
gtarpenning Oct 13, 2022
af1a38a
does local resource change CI success?
gtarpenning Oct 13, 2022
a1cf26e
intentionally raise exception to see worker error
gtarpenning Oct 13, 2022
2890b0d
circle circle circle circle circle
gtarpenning Oct 13, 2022
b23666e
fixed tests, working?
gtarpenning Oct 18, 2022
aae38f2
merge
gtarpenning Oct 18, 2022
e1e13d1
non deterministic create artifact error
gtarpenning Oct 18, 2022
c8745a4
added more time pauses and docker mocking
gtarpenning Oct 18, 2022
cc6680c
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 18, 2022
cc4fb02
wip testing with mocked git
gtarpenning Oct 19, 2022
554f1f1
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 19, 2022
9febc73
mocked git repo test
gtarpenning Oct 19, 2022
191cce3
use mocked_fetchable_git_repo instead of actual wandb examples
KyleGoyette Oct 19, 2022
02c43aa
now mocking just the validation step
gtarpenning Oct 19, 2022
1954b42
updated test
gtarpenning Oct 19, 2022
eaf91ce
no more tests with teeth
gtarpenning Oct 19, 2022
1d2cf65
literally no teeth
gtarpenning Oct 20, 2022
a4fe7ce
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 20, 2022
3de3f4c
circle dance!
gtarpenning Oct 20, 2022
03f91c2
revert to old wandb.init call, outside runner
gtarpenning Oct 20, 2022
c9f8a36
technically most correct test setup, does this really fail?
gtarpenning Oct 31, 2022
e4e8432
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Oct 31, 2022
c9a79d8
added sleeps to tests, only one failing reliably
gtarpenning Oct 31, 2022
6c86cfb
added sleeps to tests, only one failing reliably
gtarpenning Oct 31, 2022
05c4aa0
slightly changed testing apparatus to hopefully circumvent conflictin…
gtarpenning Oct 31, 2022
2bdab3e
slightly changed testing apparatus to hopefully circumvent conflictin…
gtarpenning Oct 31, 2022
b1512e0
wip tests
gtarpenning Nov 1, 2022
5db8098
improved codcov
gtarpenning Nov 7, 2022
3e45f21
improved codecov
gtarpenning Nov 7, 2022
ed29a75
lint
gtarpenning Nov 7, 2022
0f76282
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Nov 7, 2022
3871859
remove docker support
gtarpenning Nov 7, 2022
de9b8b5
test(artifacts): skip flaky test :(
speezepearson Nov 8, 2022
d744052
Merge branch 'spencerpearson/skip' of github.com:wandb/client into la…
gtarpenning Nov 8, 2022
701104b
remove unused import
gtarpenning Nov 8, 2022
6dcb70f
removed api call
gtarpenning Nov 8, 2022
21b00b7
removed proj creation from test
gtarpenning Nov 8, 2022
e502de1
updated api key test
gtarpenning Nov 8, 2022
b43b3e6
fixed test
gtarpenning Nov 8, 2022
52688c4
Merge branch 'main' into griffin/launch/update-old-api-key-test
gtarpenning Nov 11, 2022
12ab3a2
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Nov 11, 2022
ac43e36
merge with fix
gtarpenning Nov 11, 2022
44d5d9b
cleaned up tests
gtarpenning Nov 14, 2022
58baa47
improved testing coverage
gtarpenning Nov 14, 2022
e962039
Merge github.com:wandb/client into launch-build-on-queue
gtarpenning Nov 14, 2022
d5d4270
patched docker push
gtarpenning Nov 14, 2022
0fbd51b
patched docker push
gtarpenning Nov 14, 2022
3f1fa97
docker build mock
gtarpenning Nov 14, 2022
81c5c2b
updated lambda
gtarpenning Nov 14, 2022
f4d66e3
one more testing patch
gtarpenning Nov 14, 2022
bbcf958
remove comment
gtarpenning Nov 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
100 changes: 81 additions & 19 deletions tests/unit_tests/tests_launch/test_launch.py
@@ -1,36 +1,98 @@
import os

import pytest
import wandb
from wandb.errors import LaunchError
from wandb.sdk.internal.internal_api import Api as InternalApi
from wandb.sdk.launch.launch_add import launch_add
from wandb.sdk.launch.launch import run


def test_launch_delete_queued_run(relay_server, runner, user, monkeypatch):
def test_launch_repository(
relay_server, runner, user, monkeypatch, wandb_init, test_settings
):
queue = "default"
proj = "test"
proj = "test1"
repo = "testing123"
uri = "https://github.com/wandb/examples.git"
entry_point = ["python", "/examples/examples/launch/launch-quickstart/train.py"]

settings = test_settings({"project": proj})
api = InternalApi()
os.environ["WANDB_PROJECT"] = proj # required for artifact query

# create project
run = wandb.init(project=proj)
run.finish()
monkeypatch.setattr(
wandb.sdk.launch.builder.build,
"validate_docker_installation",
lambda: None,
)

monkeypatch.setattr(
wandb.docker,
"build",
lambda tags, file, context_path: None,
)

def patched_docker_push(reg, tag):
assert reg == repo

monkeypatch.setattr(
wandb.docker,
"push",
lambda reg, tag: patched_docker_push(reg, tag),
)

with relay_server():
wandb_init(settings=settings).finish()
api.create_run_queue(
entity=user, project=proj, queue_name=queue, access="PROJECT"
)

queued_run = launch_add(
uri=uri,
entity=user,
project=proj,
queue_name=queue,
entry_point=entry_point,
)
with pytest.raises(LaunchError) as e_info:
run(
api,
uri=uri,
entity=user,
project=proj,
entry_point=entry_point,
repository=repo,
)

assert "Failed to push image to repository" in str(e_info)


def test_launch_incorrect_backend(
relay_server, runner, user, monkeypatch, wandb_init, test_settings
):
proj = "test1"
uri = "https://github.com/wandb/examples.git"
entry_point = ["python", "/examples/examples/launch/launch-quickstart/train.py"]
settings = test_settings({"project": proj})
api = InternalApi()

monkeypatch.setattr(
"wandb.sdk.launch.launch.fetch_and_validate_project",
lambda _1, _2: "something",
)

monkeypatch.setattr(
wandb.sdk.launch.builder.build,
"validate_docker_installation",
lambda: None,
)

monkeypatch.setattr(
"wandb.docker",
lambda: None,
)

with relay_server():
r = wandb_init(settings=settings)

assert queued_run.state == "pending"
with pytest.raises(LaunchError) as e_info:
run(
api,
uri=uri,
entity=user,
project=proj,
entry_point=entry_point,
resource="testing123",
)

queued_run.delete()
assert "Resource name not among available resources" in str(e_info)
r.finish()