[nomerge] Use ASM 9.3, enabling JDK 19 support #10000

Philippus · 2022-04-13T12:54:37Z

Modeled after #9701 and #9916. Fixes scala/bug#12570 for scala 2.12.x.
nomerge, because there's a separate 2.13.x PR: #10001

~~I think there's first some work to be done in https://github.com/scala/scala-asm.~~

He-Pin · 2022-04-14T07:09:19Z

It's the 10000th PR!

SethTisue

thanks Philippus! and thanks Lukas for doing the scala-asm part

### What changes were proposed in this pull request? This PR aims to upgrade Scala to 2.12.16 ### Why are the changes needed? This version bring some bug fix and start to try to support Java 19 scala/scala@v2.12.15...v2.12.16 - [Upgrade to asm 9.3, for JDK19 support](scala/scala#10000) - [Fix codegen for MH.invoke etc under JDK 17 -release](scala/scala#9930) - [Deprecation related SecurityManager on JDK 17 ](scala/scala#9775) ### Does this PR introduce _any_ user-facing change? Yes, this is a Scala version change. ### How was this patch tested? Pass Github Actions Closes #36807 from LuciferYang/SPARK-39414. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

* [SPARK-31168][BUILD] Upgrade Scala to 2.12.14 ### What changes were proposed in this pull request? This PR is the 4th try to upgrade Scala 2.12.x in order to see the feasibility. - #27929 (Upgrade Scala to 2.12.11, wangyum ) - #30940 (Upgrade Scala to 2.12.12, viirya ) - #31223 (Upgrade Scala to 2.12.13, dongjoon-hyun ) Note that Scala 2.12.14 has the following fix for Apache Spark community. - Fix cyclic error in runtime reflection (protobuf), a regression that prevented Spark upgrading to 2.12.13 REQUIREMENTS: - [x] `silencer` library is released via ghik/silencer#66 - [x] `genjavadoc` library is released via lightbend/genjavadoc#282 ### Why are the changes needed? Apache Spark was stuck to 2.12.10 due to the regression in Scala 2.12.11/2.12.12/2.12.13. This will bring all the bug fixes. - https://github.com/scala/scala/releases/tag/v2.12.14 - https://github.com/scala/scala/releases/tag/v2.12.13 - https://github.com/scala/scala/releases/tag/v2.12.12 - https://github.com/scala/scala/releases/tag/v2.12.11 ### Does this PR introduce _any_ user-facing change? Yes, but this is a bug-fixed version. ### How was this patch tested? Pass the CIs. Closes #32697 from dongjoon-hyun/SPARK-31168. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> (cherry picked from commit 6c4b60f) * [SPARK-31168][BUILD] Upgrade Scala to 2.12.14 ### What changes were proposed in this pull request? This PR is the 4th try to upgrade Scala 2.12.x in order to see the feasibility. - #27929 (Upgrade Scala to 2.12.11, wangyum ) - #30940 (Upgrade Scala to 2.12.12, viirya ) - #31223 (Upgrade Scala to 2.12.13, dongjoon-hyun ) Note that Scala 2.12.14 has the following fix for Apache Spark community. - Fix cyclic error in runtime reflection (protobuf), a regression that prevented Spark upgrading to 2.12.13 REQUIREMENTS: - [x] `silencer` library is released via ghik/silencer#66 - [x] `genjavadoc` library is released via lightbend/genjavadoc#282 ### Why are the changes needed? Apache Spark was stuck to 2.12.10 due to the regression in Scala 2.12.11/2.12.12/2.12.13. This will bring all the bug fixes. - https://github.com/scala/scala/releases/tag/v2.12.14 - https://github.com/scala/scala/releases/tag/v2.12.13 - https://github.com/scala/scala/releases/tag/v2.12.12 - https://github.com/scala/scala/releases/tag/v2.12.11 ### Does this PR introduce _any_ user-facing change? Yes, but this is a bug-fixed version. ### How was this patch tested? Pass the CIs. Closes #32697 from dongjoon-hyun/SPARK-31168. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> (cherry picked from commit 6c4b60f) * [SPARK-36759][BUILD] Upgrade Scala to 2.12.15 ### What changes were proposed in this pull request? This PR aims to upgrade Scala to 2.12.15 to support Java 17/18 better. ### Why are the changes needed? Scala 2.12.15 improves compatibility with JDK 17 and 18: https://github.com/scala/scala/releases/tag/v2.12.15 - Avoids IllegalArgumentException in JDK 17+ for lambda deserialization - Upgrades to ASM 9.2, for JDK 18 support in optimizer ### Does this PR introduce _any_ user-facing change? Yes, this is a Scala version change. ### How was this patch tested? Pass the CIs Closes #33999 from dongjoon-hyun/SPARK-36759. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 16f1f71) * [SPARK-36759][BUILD][FOLLOWUP] Update version in scala-2.12 profile and doc ### What changes were proposed in this pull request? This is a follow-up to fix the leftover during switching the Scala version. ### Why are the changes needed? This should be consistent. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? This is not tested by UT. We need to check manually. There is no more `2.12.14`. ``` $ git grep 2.12.14 R/pkg/tests/fulltests/test_sparkSQL.R: c(as.Date("2012-12-14"), as.Date("2013-12-15"), as.Date("2014-12-16"))) data/mllib/ridge-data/lpsa.data:3.5307626,0.987291634724086 -0.36279314978779 -0.922212414640967 0.232904453212813 -0.522940888712441 1.79270085261407 0.342627053981254 1.26288870310799 sql/hive/src/test/resources/data/files/over10k:-3|454|65705|4294967468|62.12|14.32|true|mike white|2013-03-01 09:11:58.703087|40.18|joggying ``` Closes #34020 from dongjoon-hyun/SPARK-36759-2. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit adbea25) * [SPARK-39414][BUILD] Upgrade Scala to 2.12.16 ### What changes were proposed in this pull request? This PR aims to upgrade Scala to 2.12.16 ### Why are the changes needed? This version bring some bug fix and start to try to support Java 19 scala/scala@v2.12.15...v2.12.16 - [Upgrade to asm 9.3, for JDK19 support](scala/scala#10000) - [Fix codegen for MH.invoke etc under JDK 17 -release](scala/scala#9930) - [Deprecation related SecurityManager on JDK 17 ](scala/scala#9775) ### Does this PR introduce _any_ user-facing change? Yes, this is a Scala version change. ### How was this patch tested? Pass Github Actions Closes #36807 from LuciferYang/SPARK-39414. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit ed875a8) * fix * fix * fix * fix �[0m[�[31merror�[0m] �[0m[warn] /home/jenkins/workspace/spark-sql-catalyst-3.0/core/src/main/scala/org/apache/spark/scheduler/SpillableTaskResultGetter.scala:36: non-variable type argument org.apache.spark.scheduler.DirectTaskResult[_] in type pattern scala.runtime.NonLocalReturnControl[org.apache.spark.scheduler.DirectTaskResult[_]] is unchecked since it is eliminated by erasure�[0m �[0m[�[31merror�[0m] �[0m[warn] private[spark] class SpillableTaskResultGetter(sparkEnv: SparkEnv, scheduler: TaskSchedulerImpl)�[0m �[0m[�[31merror�[0m] �[0m[warn] �[0m * fix �[0m[�[31merror�[0m] �[0m[warn] /home/jenkins/workspace/spark-sql-catalyst-3.0/mllib/src/main/scala/org/apache/spark/ml/feature/RFormulaParser.scala:287: match may not be exhaustive.�[0m �[0m[�[31merror�[0m] �[0mIt would fail on the following input: ~(~(_, (x: String forSome x not in "^")), _)�[0m �[0m[�[31merror�[0m] �[0m[warn] private val pow: Parser[Term] = term ~ "^" ~ "^[1-9]\\d*".r ^^ {�[0m �[0m[�[31merror�[0m] �[0m[warn] �[0m �[0m[�[31merror�[0m] �[0m[warn] /home/jenkins/workspace/spark-sql-catalyst-3.0/mllib/src/main/scala/org/apache/spark/ml/feature/RFormulaParser.scala:301: match may not be exhaustive.�[0m �[0m[�[31merror�[0m] �[0mIt would fail on the following input: ~(~(_, (x: String forSome x not in "~")), _)�[0m �[0m[�[31merror�[0m] �[0m[warn] (label ~ "~" ~ expr) ^^ { case r ~ "~" ~ t => ParsedRFormula(r, t.asTerms.terms) }�[0m �[0m[�[31merror�[0m] �[0m[warn] �[0m * fix Co-authored-by: Dongjoon Hyun <dhyun@apple.com> Co-authored-by: Dongjoon Hyun <dongjoon@apache.org> Co-authored-by: yangjie01 <yangjie01@baidu.com>

scala-jenkins added this to the 2.12.16 milestone Apr 13, 2022

This was referenced Apr 13, 2022

Support JDK 19 #10001

Merged

Update asm to 9.3 scala/scala3#14926

Merged

SethTisue marked this pull request as draft April 13, 2022 19:45

SethTisue self-assigned this Apr 13, 2022

Upgrade to asm 9.3, for JDK19 support

982d925

Philippus force-pushed the issue-12570-scala212 branch from 9e18f96 to 982d925 Compare April 21, 2022 14:53

lrytz marked this pull request as ready for review April 21, 2022 18:49

lrytz requested a review from SethTisue April 21, 2022 18:49

SethTisue added the release-notes worth highlighting in next release notes label Apr 21, 2022

SethTisue approved these changes Apr 21, 2022

View reviewed changes

SethTisue merged commit 137827f into scala:2.12.x Apr 21, 2022

Philippus deleted the issue-12570-scala212 branch April 21, 2022 19:39

SethTisue mentioned this pull request Apr 22, 2022

[backport] Make -target support JDK 8 through 19 (and deprecate 5 through 7) #9916

Merged

SethTisue mentioned this pull request May 9, 2022

Scalac can't generate 55(jvm11) bytecode version scala/bug#12588

Closed

SethTisue changed the title ~~[nomerge] Upgrade to asm 9.3, for JDK19 support~~ [nomerge] For JDK 19 support, use asm 9.3 May 13, 2022

SethTisue changed the title ~~[nomerge] For JDK 19 support, use asm 9.3~~ [nomerge] Use ASM 9.3, enabling JDK 19 support May 16, 2022

LuciferYang mentioned this pull request Jun 9, 2022

[SPARK-39414][BUILD] Upgrade Scala to 2.12.16 apache/spark#36807

Closed

mdedetrich mentioned this pull request Jun 17, 2022

KAFKA-13980: Upgrade from Scala 2.12.15 to 2.12.17 apache/kafka#12284

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[nomerge] Use ASM 9.3, enabling JDK 19 support #10000

[nomerge] Use ASM 9.3, enabling JDK 19 support #10000

Philippus commented Apr 13, 2022 •

edited by SethTisue

He-Pin commented Apr 14, 2022

SethTisue left a comment •

edited

[nomerge] Use ASM 9.3, enabling JDK 19 support #10000

[nomerge] Use ASM 9.3, enabling JDK 19 support #10000

Conversation

Philippus commented Apr 13, 2022 • edited by SethTisue

He-Pin commented Apr 14, 2022

SethTisue left a comment • edited

Choose a reason for hiding this comment

Philippus commented Apr 13, 2022 •

edited by SethTisue

SethTisue left a comment •

edited