Skip to content

Commit

Permalink
[SPARK-41830][CONNECT][PYTHON][TESTS][FOLLOWUP] Enable parity test `t…
Browse files Browse the repository at this point in the history
…est_sample`

### What changes were proposed in this pull request?
Enable parity test `test_sample`

### Why are the changes needed?
For test coverage

### Does this PR introduce _any_ user-facing change?
no, test-only

### How was this patch tested?
enabled test

Closes #39765 from zhengruifeng/connect_enable_41830.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
zhengruifeng authored and dongjoon-hyun committed Jan 29, 2023
1 parent 0e46106 commit 2fa1d6b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 6 deletions.
5 changes: 0 additions & 5 deletions python/pyspark/sql/tests/connect/test_parity_dataframe.py
Expand Up @@ -95,11 +95,6 @@ def test_require_cross(self):
def test_same_semantics_error(self):
super().test_same_semantics_error()

# TODO(SPARK-41830): Fix DataFrame.sample parameters
@unittest.skip("Fails in Spark Connect, should enable.")
def test_sample(self):
super().test_sample()

@unittest.skip("Spark Connect does not support RDD but the tests depend on them.")
def test_toDF_with_schema_string(self):
super().test_toDF_with_schema_string()
Expand Down
6 changes: 5 additions & 1 deletion python/pyspark/sql/tests/test_dataframe.py
Expand Up @@ -46,6 +46,7 @@
from pyspark.errors import (
AnalysisException,
IllegalArgumentException,
SparkConnectException,
SparkConnectAnalysisException,
)
from pyspark.testing.sqlutils import (
Expand Down Expand Up @@ -888,7 +889,10 @@ def test_sample(self):

self.assertRaises(TypeError, lambda: self.spark.range(1).sample(seed="abc"))

self.assertRaises(IllegalArgumentException, lambda: self.spark.range(1).sample(-1.0))
self.assertRaises(
(IllegalArgumentException, SparkConnectException),
lambda: self.spark.range(1).sample(-1.0).count(),
)

def test_toDF_with_schema_string(self):
data = [Row(key=i, value=str(i)) for i in range(100)]
Expand Down

0 comments on commit 2fa1d6b

Please sign in to comment.