Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use xfail consistently for known failing tests #867

Merged
merged 36 commits into from Jan 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
bfa990d
Uncomment tests in test_compatibility.py
charlesbluca Oct 18, 2022
4c3c9ad
Uncomments tests in test_explain.py
charlesbluca Oct 18, 2022
3c6f404
xfail instead of skipping for failing independent cluster tests
charlesbluca Oct 19, 2022
32997cc
Switch skips to xfails in test_groupby.py
charlesbluca Oct 19, 2022
cebc475
Uncomments tests in test_over.py
charlesbluca Oct 19, 2022
3395a02
Switch skips to xfails in test_postgres.py
charlesbluca Oct 19, 2022
74bedf7
Uncomments tests in test_rex.py
charlesbluca Oct 19, 2022
d57e949
Switch skips to xfails in test_schema.py
charlesbluca Oct 19, 2022
9fb367b
Switch skips to xfails in test_jdbc.py
charlesbluca Oct 19, 2022
fd50a46
Switch skips to xfails in test_sample.py
charlesbluca Oct 19, 2022
2f5fb4e
Switch skips to xfails in test_server.py
charlesbluca Oct 19, 2022
2177378
Uncomments tests in test_context.py
charlesbluca Oct 19, 2022
f0bf4d8
Un-xfail passing tests
charlesbluca Oct 20, 2022
3cdaa46
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Oct 20, 2022
2cfdbf7
Add xfails to some broken tests
charlesbluca Oct 24, 2022
349b7fc
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Oct 24, 2022
86a72a5
xfail alter table/schema tests
charlesbluca Oct 24, 2022
1f74895
Un-xfail test_regr_aggregation on GPU
charlesbluca Oct 24, 2022
75db170
Merge branch 'main' into uncommment-tests
charlesbluca Oct 25, 2022
479e9e0
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Oct 26, 2022
890a417
Fix external scheduler xfail
charlesbluca Oct 26, 2022
90a4310
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Oct 31, 2022
658354f
Merge branch 'main' into uncommment-tests
charlesbluca Nov 2, 2022
5f5a960
Merge branch 'main' into uncommment-tests
ayushdg Nov 30, 2022
49d0f72
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Dec 13, 2022
d9ecc75
Replace skip_if_external_scheduler in ML wrapper tests
charlesbluca Dec 13, 2022
4ff69ae
unxfail test_date_functions
charlesbluca Dec 13, 2022
0035383
Set max constraint for fastapi
charlesbluca Dec 14, 2022
37dc53b
Skip test_iterative_and_prediction on cluster
charlesbluca Dec 14, 2022
07dfb41
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Jan 5, 2023
cc10500
Fix style errors
charlesbluca Jan 5, 2023
af7830d
Remove uses of skip_if_external_scheduler
charlesbluca Jan 5, 2023
8783b84
Resolve remaining cluster tests
charlesbluca Jan 9, 2023
6fd50f2
Merge branch 'main' into uncommment-tests
charlesbluca Jan 9, 2023
ee0e689
Merge remote-tracking branch 'origin/main' into uncommment-tests
charlesbluca Jan 9, 2023
96434b0
un-xfail fugue tests
charlesbluca Jan 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion continuous_integration/environment-3.10-dev.yaml
Expand Up @@ -4,7 +4,8 @@ channels:
- nodefaults
dependencies:
- dask>=2022.3.0
- fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
- fastapi>=0.69.0,<0.87.0
Comment on lines +7 to +8
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastapi>=0.87.0 pulls in starlette>=0.21.0, which switched to using httpx instead of requests for its TestClient API (encode/starlette#1376); it's not immediately obvious to me how to support both versions of the API, so this pins us to the older starlette packages for now.

- fugue>=0.7.3
- intake>=0.6.0
- jsonschema
Expand Down
3 changes: 2 additions & 1 deletion continuous_integration/environment-3.9-dev.yaml
Expand Up @@ -4,7 +4,8 @@ channels:
- nodefaults
dependencies:
- dask>=2022.3.0
- fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
- fastapi>=0.69.0,<0.87.0
- fugue>=0.7.3
- intake>=0.6.0
- jsonschema
Expand Down
3 changes: 2 additions & 1 deletion continuous_integration/gpuci/environment.yaml
Expand Up @@ -7,7 +7,8 @@ channels:
- nodefaults
dependencies:
- dask>=2022.3.0
- fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
- fastapi>=0.69.0,<0.87.0
- fugue>=0.7.3
- intake>=0.6.0
- jsonschema
Expand Down
3 changes: 2 additions & 1 deletion continuous_integration/recipe/meta.yaml
Expand Up @@ -32,7 +32,8 @@ requirements:
- python
- dask >=2022.3.0
- pandas >=1.4.0
- fastapi >=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
- fastapi >=0.69.0,<0.87.0
- uvicorn >=0.13.4
- tzlocal >=2.1
- prompt-toolkit >=3.0.8
Expand Down
3 changes: 2 additions & 1 deletion docker/conda.txt
Expand Up @@ -10,7 +10,8 @@ pytest-xdist
mock>=4.0.3
sphinx>=3.2.1
tzlocal>=2.1
fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
fastapi>=0.69.0,<0.87.0
nest-asyncio>=1.4.3
uvicorn>=0.13.4
pyarrow>=6.0.1
Expand Down
3 changes: 2 additions & 1 deletion docker/main.dockerfile
Expand Up @@ -18,7 +18,8 @@ RUN mamba install -y \
# core dependencies
"dask>=2022.3.0" \
"pandas>=1.4.0" \
"fastapi>=0.69.0" \
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
"fastapi>=0.69.0,<0.87.0" \
"uvicorn>=0.13.4" \
"tzlocal>=2.1" \
"prompt_toolkit>=3.0.8" \
Expand Down
3 changes: 2 additions & 1 deletion docs/environment.yml
Expand Up @@ -11,7 +11,8 @@ dependencies:
- pandas>=1.4.0
- fugue>=0.7.3
- jpype1>=1.0.2
- fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
- fastapi>=0.69.0,<0.87.0
- uvicorn>=0.13.4
- tzlocal>=2.1
- prompt_toolkit>=3.0.8
Expand Down
3 changes: 2 additions & 1 deletion docs/requirements-docs.txt
Expand Up @@ -4,7 +4,8 @@ dask-sphinx-theme>=3.0.0
dask>=2022.3.0
pandas>=1.4.0
fugue>=0.7.3
fastapi>=0.69.0
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
fastapi>=0.69.0,<0.87.0
uvicorn>=0.13.4
tzlocal>=2.1
prompt_toolkit>=3.0.8
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Expand Up @@ -44,7 +44,8 @@
install_requires=[
"dask[dataframe,distributed]>=2022.3.0",
"pandas>=1.4.0",
"fastapi>=0.69.0",
# FIXME: handling is needed for httpx-based fastapi>=0.87.0
"fastapi>=0.69.0,<0.87.0",
"uvicorn>=0.13.4",
"tzlocal>=2.1",
"prompt_toolkit>=3.0.8",
Expand Down
6 changes: 3 additions & 3 deletions tests/integration/fixtures.py
Expand Up @@ -332,15 +332,15 @@ def gpu_client(gpu_cluster):
# client for all computations. otherwise, only connect to a client
# when specified.
@pytest.fixture(
scope="function" if SCHEDULER_ADDR is None else "session",
scope="function",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we use a session-wide client for external cluster testing, it ends up getting blown away with dask.config.refresh() while running test_config.py. Since there wasn't any immediate way to avoid this, I made this change so that we reconnect to the external cluster on each test, so that tests run after test_config.py are still able to use the external cluster as expected.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wonder if test_config can be improved somehow to save the state of configs before the tests start and restore that state at the end of the test.

Either way I didn't see a noticeable change in overall test runtimes with this change so it might be okay to improve later.

autouse=False if SCHEDULER_ADDR is None else True,
)
def client():
with Client(address=SCHEDULER_ADDR) as client:
yield client


skip_if_external_scheduler = pytest.mark.skipif(
os.getenv("DASK_SQL_TEST_SCHEDULER", None) is not None,
xfail_if_external_scheduler = pytest.mark.xfail(
condition=os.getenv("DASK_SQL_TEST_SCHEDULER", None) is not None,
reason="Can not run with external cluster",
)