Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

trino-hive tests sometimes fail with OutOfMemoryError #11848

Closed
homar opened this issue Apr 7, 2022 · 6 comments · Fixed by #13411
Closed

trino-hive tests sometimes fail with OutOfMemoryError #11848

homar opened this issue Apr 7, 2022 · 6 comments · Fixed by #13411

Comments

@homar
Copy link
Member

homar commented Apr 7, 2022 

2022-04-06T21:54:38.5790757Z [ERROR] java.lang.OutOfMemoryError: Java heap space
2022-04-06T21:55:08.8826395Z [ERROR] Terminating due to java.lang.OutOfMemoryError: Java heap space
2022-04-06T21:55:09.9314818Z [INFO] 
2022-04-06T21:55:09.9328385Z [INFO] Results:
2022-04-06T21:55:09.9339673Z [INFO] 
2022-04-06T21:55:09.9639461Z [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
2022-04-06T21:55:09.9651254Z [INFO] 
2022-04-06T21:55:10.0021123Z [WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /home/runner/work/trino/trino/plugin/trino-hive/target/surefire-reports/2022-04-06T21-29-07_236-jvmRun1.dumpstream
2022-04-06T21:55:10.3164810Z [INFO] ------------------------------------------------------------------------
2022-04-06T21:55:10.3176202Z [INFO] BUILD FAILURE
2022-04-06T21:55:10.3185261Z [INFO] ------------------------------------------------------------------------
2022-04-06T21:55:10.3228510Z [INFO] Total time:  26:07 min
2022-04-06T21:55:10.3257596Z [INFO] Finished at: 2022-04-06T21:55:10Z
2022-04-06T21:55:10.3258513Z [INFO] ------------------------------------------------------------------------
2022-04-06T21:55:10.4468660Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project trino-hive: There are test failures.
2022-04-06T21:55:10.4476741Z [ERROR] 
2022-04-06T21:55:10.4477766Z [ERROR] Please refer to /home/runner/work/trino/trino/plugin/trino-hive/target/surefire-reports for the individual test results.
2022-04-06T21:55:10.4492185Z [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
2022-04-06T21:55:10.4498017Z [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
2022-04-06T21:55:10.4505538Z [ERROR] Command was /bin/sh -c cd /home/runner/work/trino/trino/plugin/trino-hive && /opt/hostedtoolcache/Java_Zulu_jdk/11.0.14-9/x64/bin/java -Dfile.encoding=UTF-8 -Xmx3g -Xms3g -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -jar /home/runner/work/trino/trino/plugin/trino-hive/target/surefire/surefirebooter1158786089791957836.jar /home/runner/work/trino/trino/plugin/trino-hive/target/surefire 2022-04-06T21-29-07_236-jvmRun1 surefire9694311627023209032tmp surefire_01246237106907864530tmp
2022-04-06T21:55:10.4527656Z [ERROR] Error occurred in starting fork, check output in log
2022-04-06T21:55:10.4532891Z [ERROR] Process Exit Code: 3
2022-04-06T21:55:10.4536179Z [ERROR] Crashed tests:
2022-04-06T21:55:10.4539786Z [ERROR] io.trino.plugin.hive.TestHiveConnectorTest
2022-04-06T21:55:10.4547861Z [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
2022-04-06T21:55:10.4554367Z [ERROR] Command was /bin/sh -c cd /home/runner/work/trino/trino/plugin/trino-hive && /opt/hostedtoolcache/Java_Zulu_jdk/11.0.14-9/x64/bin/java -Dfile.encoding=UTF-8 -Xmx3g -Xms3g -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -jar /home/runner/work/trino/trino/plugin/trino-hive/target/surefire/surefirebooter1158786089791957836.jar /home/runner/work/trino/trino/plugin/trino-hive/target/surefire 2022-04-06T21-29-07_236-jvmRun1 surefire9694311627023209032tmp surefire_01246237106907864530tmp
2022-04-06T21:55:10.4566819Z [ERROR] Error occurred in starting fork, check output in log
2022-04-06T21:55:10.4576518Z [ERROR] Process Exit Code: 3
2022-04-06T21:55:10.4583695Z [ERROR] Crashed tests:
2022-04-06T21:55:10.4589820Z [ERROR] io.trino.plugin.hive.TestHiveConnectorTest
2022-04-06T21:55:10.4597713Z [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:748)
2022-04-06T21:55:10.4607247Z [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:305)
2022-04-06T21:55:10.4617092Z [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:265)
2022-04-06T21:55:10.4628734Z [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1314)
2022-04-06T21:55:10.4638140Z [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1159)
2022-04-06T21:55:10.4651534Z [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:932)
2022-04-06T21:55:10.4652622Z [ERROR] 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
2022-04-06T21:55:10.4653621Z [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
2022-04-06T21:55:10.4654508Z [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
2022-04-06T21:55:10.4655464Z [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
2022-04-06T21:55:10.4656552Z [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
2022-04-06T21:55:10.4657694Z [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
2022-04-06T21:55:10.4658931Z [ERROR] 	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
2022-04-06T21:55:10.6392657Z [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
2022-04-06T21:55:10.6405624Z [ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
2022-04-06T21:55:10.6415044Z [ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
2022-04-06T21:55:10.6415948Z [ERROR] 	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
2022-04-06T21:55:10.6417048Z [ERROR] 	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:972)
2022-04-06T21:55:10.6417742Z [ERROR] 	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
2022-04-06T21:55:10.6418267Z [ERROR] 	at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
2022-04-06T21:55:10.6418953Z [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-04-06T21:55:10.6432145Z [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-04-06T21:55:10.6490503Z [ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-04-06T21:55:10.6491752Z [ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2022-04-06T21:55:10.6500515Z [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
2022-04-06T21:55:10.6501521Z [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
2022-04-06T21:55:10.6504542Z [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
2022-04-06T21:55:10.6507706Z [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
2022-04-06T21:55:10.6516005Z [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-04-06T21:55:10.6521217Z [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-04-06T21:55:10.6526703Z [ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-04-06T21:55:10.6536780Z [ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2022-04-06T21:55:10.6540072Z [ERROR] 	at org.apache.maven.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:47)
2022-04-06T21:55:10.6544038Z [ERROR] 	at org.apache.maven.wrapper.WrapperExecutor.execute(WrapperExecutor.java:156)
2022-04-06T21:55:10.6544946Z [ERROR] 	at org.apache.maven.wrapper.MavenWrapperMain.main(MavenWrapperMain.java:72)
2022-04-06T21:55:10.6551440Z [ERROR] -> [Help 1]
2022-04-06T21:55:10.6560754Z [ERROR] 
2022-04-06T21:55:10.6563488Z [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
2022-04-06T21:55:10.6566222Z [ERROR] Re-run Maven using the -X switch to enable full debug logging.
2022-04-06T21:55:10.6566688Z [ERROR] 
2022-04-06T21:55:10.6571363Z [ERROR] For more information about the errors and possible solutions, please read the following articles:
2022-04-06T21:55:10.6577505Z [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
2022-04-06T21:55:11.0344285Z ##[error]Process completed with exit code 1.

For example here https://github.com/trinodb/trino/runs/5858114730?check_suite_focus=true
@homar homar mentioned this issue Apr 7, 2022
Merged
@findepi findepi mentioned this issue Apr 7, 2022
Closed
@findepi
Copy link
Member

findepi commented Apr 7, 2022

apache/maven-surefire#407 might help.

@nineinchnick
Copy link
Member

@findepi 3.0.0-M6 was released 3 days ago and includes that change, could you update surefire in airbase?

@nineinchnick
Copy link
Member

airlift/airbase#310

@nineinchnick nineinchnick mentioned this issue Apr 11, 2022
Closed
@sopel39 sopel39 mentioned this issue Apr 25, 2022
Merged
@sopel39
Copy link
Member

sopel39 commented Apr 25, 2022

https://github.com/trinodb/trino/runs/6110587357?check_suite_focus=true

@findepi
Copy link
Member

findepi commented May 27, 2022

https://github.com/trinodb/trino/runs/6626556392?check_suite_focus=true

@findepi findepi mentioned this issue May 27, 2022
Merged
@findepi findepi changed the title OutOfMemory: Java heap space in trino-hive trino-hive tests sometimes fail with OutOfMemoryError Jun 2, 2022
@arhimondr
Copy link
Contributor

https://github.com/trinodb/trino/runs/7369520202?check_suite_focus=true

@findepi findepi mentioned this issue Jul 29, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
5 participants
@homar @findepi @nineinchnick @arhimondr @sopel39