Skip to content

Commit

Permalink
[SPARK-41548][CONNECT][TESTS] Disable ANSI mode in pyspark.sql.tests.…
Browse files Browse the repository at this point in the history
…connect.test_connect_functions

### What changes were proposed in this pull request?

This PR proposes to disable ANSI mode in `pyspark.sql.tests.connect.test_connect_functions` temporarily.

This PR is sort of a followup of #38946 and #39071

### Why are the changes needed?

There are failures in `test_connect_functions` with ANSI mode on (https://github.com/apache/spark/actions/runs/3709431687/jobs/6288067223). I tried to fix but they are tricky to fix because Spark Connect does not respect the runtime configuration at the server side.

It is also tricky to fix the test to pass in both ANSI mode on and off. Therefore, it disables temporarily to make other tests pass. Note that PySpark tests stop in the middle if one fails.

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

Manually verified that:

```bash
SPARK_ANSI_SQL_MODE=true ./python/run-tests --modules pyspark-connect -p 1
```
passes fine.

Closes #39092 from HyukjinKwon/SPARK-41548.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
HyukjinKwon authored and dongjoon-hyun committed Dec 16, 2022
1 parent 06ec98b commit d3e7d8f
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions python/pyspark/sql/tests/connect/test_connect_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -1411,8 +1411,12 @@ def test_misc_functions(self):


if __name__ == "__main__":
import os
from pyspark.sql.tests.connect.test_connect_function import * # noqa: F401

# TODO(SPARK-41547): Enable ANSI mode in this file.
os.environ["SPARK_ANSI_SQL_MODE"] = "false"

try:
import xmlrunner # type: ignore

Expand Down

0 comments on commit d3e7d8f

Please sign in to comment.