Adjust joins cost estimation [HZ-2658, HZ-2494] #25249
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sasha Syrotenko <oleksandr.syrotenko@hazelcast.com>
Fly-Style
added
Team: SQL
SQL-only
Signed-off-by: Sasha Syrotenko <oleksandr.syrotenko@hazelcast.com>
Signed-off-by: Sasha Syrotenko <oleksandr.syrotenko@hazelcast.com>
Signed-off-by: Sasha Syrotenko <oleksandr.syrotenko@hazelcast.com>
degerhz
changed the title
k-jamroz
reviewed
Aug 31, 2023
...ast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinNestedLoopPhysicalRel.java
Outdated
Show resolved
Hide resolved
k-jamroz
reviewed
Aug 31, 2023
| double cpu = (Cost.HASH_JOIN_MULTIPLIER * leftRowCount + /* TODO: selectivity */ rightRowCount) | ||
| * Cost.JOIN_ROW_CMP_MULTIPLIER; | ||
|
|
||
| return planner.getCostFactory().makeCost(rowCount, cpu, 0.); |
There was a problem hiding this comment.
AFAIK computeSelfCost should return number of rows produced not processed.
There was a problem hiding this comment.
I agree. The number of produced rows contributes to the processing costs of upstream relations.
There was a problem hiding this comment.
... produced rows contributes to the processing costs of upstream relations.
It's not correct. Calcite's planner uses a cost calculation algorithm for local RelSet and RelSubset-s.
There was a problem hiding this comment.
You are right. Calcite uses reported self-costs, which don't rely on costs of child relations.
There was a problem hiding this comment.
Calcite uses reported self-costs, which don't rely on costs of child relations.
But self cost of rel nodes usually depends on number of rows in thier inputs.
There was a problem hiding this comment.
but the formulas in PR description are outdated
k-jamroz
reviewed
Aug 31, 2023
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinHashPhysicalRel.java
Outdated
Show resolved
Hide resolved
k-jamroz
reviewed
Aug 31, 2023
...ast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinNestedLoopPhysicalRel.java
Outdated
Show resolved
Hide resolved
burakgok
requested changes
Sep 1, 2023
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/cost/Cost.java
Outdated
Show resolved
Hide resolved
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/schema/HazelcastTable.java
Outdated
Show resolved
Hide resolved
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/schema/HazelcastTable.java
Outdated
Show resolved
Hide resolved
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/schema/HazelcastTable.java
Outdated
Show resolved
Hide resolved
| double cpu = (Cost.HASH_JOIN_MULTIPLIER * leftRowCount + /* TODO: selectivity */ rightRowCount) | ||
| * Cost.JOIN_ROW_CMP_MULTIPLIER; | ||
|
|
||
| return planner.getCostFactory().makeCost(rowCount, cpu, 0.); |
There was a problem hiding this comment.
I agree. The number of produced rows contributes to the processing costs of upstream relations.
...ast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinNestedLoopPhysicalRel.java
Outdated
Show resolved
Hide resolved
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/QueryPlanListener.java
Outdated
Show resolved
Hide resolved
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/CalciteSqlOptimizer.java
Outdated
Show resolved
Hide resolved
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] Unexpected file: .github/workflows/assign-to-test-failure.yml' -------------------------- [ERROR] Unexpected file: .github/workflows/assign-to-test-failure.yml -------------------------- |
|
The job Click to expand the log file-------------------------- ---------SUMMARY---------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] reason: class file for aQute.bnd.annotation.Resolution not found -------------------------- ---------ERRORS----------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] reason: class file for aQute.bnd.annotation.Resolution not found -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[105,71] error: cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[133,40] error: cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[138,69] error: cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseRecordStore.java:[52,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseRecordStore.java:[58,20] error: cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/TieredStorageImpl.java:[180,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseTieredRecordStore.java:[61,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project hazelcast-enterprise: Compilation failure: Compilation failure: -------------------------- [ERROR] reason: class file for aQute.bnd.annotation.Resolution not found -------------------------- [ERROR] warning: unknown enum constant Resolution.OPTIONAL -------------------------- [ERROR] reason: class file for aQute.bnd.annotation.Resolution not found -------------------------- [ERROR] warning: unknown enum constant Resolution.OPTIONAL -------------------------- [ERROR] reason: class file for aQute.bnd.annotation.Resolution not found -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[80,60] error: cannot find symbol -------------------------- [ERROR] symbol: method isPermissionPriorityGrant() -------------------------- [ERROR] location: class SecurityConfig -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[105,71] error: cannot find symbol -------------------------- [ERROR] symbol: method isDeny() -------------------------- [ERROR] location: variable permCfg of type PermissionConfig -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[133,40] error: cannot find symbol -------------------------- [ERROR] symbol: method isDeny() -------------------------- [ERROR] location: variable permissionConfig of type PermissionConfig -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/security/impl/DefaultPermissionPolicy.java:[138,69] error: cannot find symbol -------------------------- [ERROR] symbol: method isDeny() -------------------------- [ERROR] location: variable permissionConfig of type PermissionConfig -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseRecordStore.java:[52,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseRecordStore.java:[58,20] error: cannot find symbol -------------------------- [ERROR] symbol: method createRecordFactory() -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/TieredStorageImpl.java:[180,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast-enterprise/hazelcast-enterprise/src/main/java/com/hazelcast/map/impl/recordstore/EnterpriseTieredRecordStore.java:[61,4] error: method does not override or implement a method from a supertype -------------------------- [ERROR] -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-enterprise -------------------------- |
burakgok
requested changes
Sep 18, 2023
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinHashPhysicalRel.java
Outdated
Show resolved
Hide resolved
…sical/JoinHashPhysicalRel.java Co-authored-by: Burak Gök <burak.gok@hazelcast.com>
k-jamroz
reviewed
Sep 19, 2023
...ast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinNestedLoopPhysicalRel.java
Show resolved
Hide resolved
k-jamroz
reviewed
Sep 28, 2023
...ast-sql/src/main/java/com/hazelcast/jet/sql/impl/opt/physical/JoinNestedLoopPhysicalRel.java
Outdated
Show resolved
Hide resolved
|
I tried to translate join processors to cost formulas to ensure we are on the same page. Could you please verify them? SqlHashJoinPJoinScanProcessorSupplierJoinByEquiJoinProcessorSupplierJoinByPrimitiveKeyProcessorSupplierVariables0 ≤ equiColumnCount ≤ min(leftColumnCount, rightColumnCount) rightConditionCount is the number of non-join conditions belonging to the right relation. elementComparisonCost is the average combined cost of |
|
Hi, @burakgok. Thank you for your really titanic job (well done, btw), but the provided formulas in the PR are final. |
k-jamroz
reviewed
Oct 18, 2023
hazelcast-sql/src/main/java/com/hazelcast/jet/sql/impl/schema/HazelcastSchemaUtils.java
Outdated
Show resolved
Hide resolved
k-jamroz
reviewed
Oct 18, 2023
| double projectionCost = getRight().getRowType().getFieldCount(); | ||
|
|
||
| double processedRowsCount = leftRowCount + rightRowCount; | ||
| double cpu = leftRowCount * Cost.HASH_JOIN_MULTIPLIER |
There was a problem hiding this comment.
in case of outer join one of the sides is broadcast. This requires more processing. Shouldn't we include that in cost calculation (I don't mean network - however that would nice too, but CPU)?
There was a problem hiding this comment.
Shouldn't we include that in cost calculation
Yes, makes sense.
There was a problem hiding this comment.
This is not what I meant: in case of outer non-equi join, outer side is broadcast. So we do rightCount*memberCount lookups which directly translates to CPU cost of lookups. However, they are also different than in case of equi-join. Eg. for query like select * from m1 left join m2 on m1.__key<>m2.__key hashmap in SqlHashJoinP degenerates to single key, and we iterate over all rows (this is a de-facto nested loop with spooling left side in memory)
Additionally, those rows have to be sent over network, but AFAIR we do not add network cost to CPU cost. We have separate network cost, which has it's own mutliplier but is generally 0.
There was a problem hiding this comment.
hashmap in SqlHashJoinP degenerates to single key
If I see correctly in case of non-equi hash join the hashmap is always degenerate. So we:
- build
memberCountsingle-unique-key hash maps (multimaps to be precise) each containingrightCountitems - then we do
leftCountlookups (in single-key hash map) and iterate over all found rows (ie.rightCountrows). This givesleftCountlookups (are they constant time with regard torightCount?) andleftCount*rightCountcomparisons for iterating over elements found in multimap (ie. all right side rows). - then projection of matching rows
There was a problem hiding this comment.
equi-join case is somewhere on the continuum between "hashmap lookup gives always single right row" and "hashmap lookup gives all right rows" depending on selectivity of equijoin columns (which we cannot estimate because we do not have column histograms).
also in equi-join hash table is partitioned, but might be skewed or even degenerate for the same reason
k-jamroz
reviewed
Oct 19, 2023
| * <li> R is a count of right side rows. | ||
| * <li> PD is a produced row count: L * R * (join selectivity). | ||
| * <li> PR is a processed row count: L * k * R, where k is 1 for non-equi-join, | ||
| * (join selectivity) ≤ k ≤ 1 for equi-join and 1/R for key lookup. |
There was a problem hiding this comment.
where is "(join selectivity) ≤ k ≤ 1 for equi-join" and "1/R for key lookup" implemented? The formula in the code is double processedRowCount = Math.max(1.0, leftRowCount * rightRowCount);
k-jamroz
reviewed
Oct 19, 2023
| @@ -56,7 +57,7 @@ public static HazelcastSchema createCatalog(Schema schema) { | |||
| * | |||
| * @return Top-level schema. | |||
| */ | |||
| public static HazelcastSchema createRootSchema(SqlCatalog catalog) { | |||
| public static HazelcastSchema createRootSchema(HazelcastInstance hz, SqlCatalog catalog) { | |||
There was a problem hiding this comment.
we do not need hz anymore
|
The job Click to expand the log file-------------------------- ---------SUMMARY---------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- ---------ERRORS----------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.12.1:testCompile (default-testCompile) on project hazelcast-sql: Compilation failure: Compilation failure: -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.UnparseTest -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.ParserNameResolutionTest -------------------------- [ERROR] -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:19:8: Unused import - com.google.common.collect.ImmutableList. [UnusedImports] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:21:8: Unused import - com.hazelcast.jet.sql.impl.connector.test.TestAbstractSqlConnector. [UnusedImports] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:22:8: Unused import - com.hazelcast.jet.sql.impl.connector.test.TestStreamSqlConnector. [UnusedImports] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:33:8: Unused import - org.junit.Ignore. [UnusedImports] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:40:15: Unused import - com.hazelcast.sql.impl.type.QueryDataTypeFamily.INTEGER. [UnusedImports] -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:41:15: Unused import - com.hazelcast.sql.impl.type.QueryDataTypeFamily.TIMESTAMP. [UnusedImports] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.1:checkstyle (default) on project hazelcast-sql: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 6 errors reported by Checkstyle 9.3 with /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-compiler/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
|
The job Click to expand the log file---------ERRORS----------- -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:19:8: Unused import - com.google.common.collect.ImmutableList. [UnusedImports] -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:21:8: Unused import - com.hazelcast.jet.sql.impl.connector.test.TestAbstractSqlConnector. [UnusedImports] -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:22:8: Unused import - com.hazelcast.jet.sql.impl.connector.test.TestStreamSqlConnector. [UnusedImports] -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:33:8: Unused import - org.junit.Ignore. [UnusedImports] -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:40:15: Unused import - com.hazelcast.sql.impl.type.QueryDataTypeFamily.INTEGER. [UnusedImports] -------------------------- [ERROR] /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/opt/physical/PhysicalJoinTest.java:41:15: Unused import - com.hazelcast.sql.impl.type.QueryDataTypeFamily.TIMESTAMP. [UnusedImports] -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.1:checkstyle (default) on project hazelcast-sql: An error has occurred in Checkstyle report generation. Failed during checkstyle execution: There are 6 errors reported by Checkstyle 9.3 with /appdisk/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/checkstyle/checkstyle.xml ruleset. -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
|
The job Click to expand the log file-------------------------- ---------SUMMARY---------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- ---------ERRORS----------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.12.1:testCompile (default-testCompile) on project hazelcast-sql: Compilation failure: Compilation failure: -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.UnparseTest -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-EE-compiler/hazelcast/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.ParserNameResolutionTest -------------------------- [ERROR] -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
|
The job Click to expand the log file-------------------------- ---------SUMMARY---------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- ---------ERRORS----------- -------------------------- [ERROR] COMPILATION ERROR : -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.12.1:testCompile (default-testCompile) on project hazelcast-sql: Compilation failure: Compilation failure: -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/ParserNameResolutionTest.java:[226,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.ParserNameResolutionTest -------------------------- [ERROR] /home/jenkins/jenkins_slave/workspace/Hazelcast-pr-builder-sql/hazelcast-sql/src/test/java/com/hazelcast/jet/sql/impl/parse/UnparseTest.java:[99,17] cannot find symbol -------------------------- [ERROR] symbol: method instance() -------------------------- [ERROR] location: class com.hazelcast.jet.sql.impl.parse.UnparseTest -------------------------- [ERROR] -> [Help 1] -------------------------- [ERROR] -------------------------- [ERROR] Re-run Maven using the -X switch to enable full debug logging. -------------------------- [ERROR] -------------------------- [ERROR] For more information about the errors and possible solutions, please read the following articles: -------------------------- [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException -------------------------- [ERROR] -------------------------- [ERROR] After correcting the problems, you can resume the build with the command -------------------------- [ERROR] mvn -rf :hazelcast-sql -------------------------- |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces adjustments to our JOIN algorithms cost estimation model. Previously, we had a pretty inaccurate model, where we chose Hash Join only when the LHS join source was an IMap, and even in not every case. After the update, the cost estimation will use a more realistic assessment according to the specifics of each JOIN algorithm.
Common metrics:
Lis a count of left-side rows.Ris a count of right-side rows.New formula to estimate NL Joins cost :
PDis a produced row count:L * R * (join selectivity).PRis processed row count:L * k * R, where k is 1 for non-equi-join,(join selectivity) ≤ k ≤ 1for equi-join and 1/R for key lookup.CPU cost is estimated as
PR * (row comparison cost) + PD * (selectivity) * (row join cost) + ((L - 1) * cost of right side scan)New formula to estimate Hash Joins:
Processed row count (PR) is L + R because we traverse both sides once per join.
CPU is
L * (hash table build cost) + R * (row comparison cost) + (PR * (row projection cost))