Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query7: fails in spark on streaming mode (not PCollectionView in streaming mode) #53

Open
echauchot opened this issue Apr 28, 2017 · 0 comments

Comments

@echauchot
Copy link

Exception in thread "main" java.lang.IllegalStateException: No TransformEvaluator registered for UNBOUNDED transform class org.apache.beam.sdk.transforms.View$CreatePCollectionView
	at com.google.common.base.Preconditions.checkState(Preconditions.java:518)
	at org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$Translator.translateUnbounded(StreamingTransformTranslator.java:529)
	at org.apache.beam.runners.spark.SparkRunner$Evaluator.translate(SparkRunner.java:435)
	at org.apache.beam.runners.spark.SparkRunner$Evaluator.doVisitTransform(SparkRunner.java:405)
	at org.apache.beam.runners.spark.SparkRunner$Evaluator.visitPrimitiveTransform(SparkRunner.java:395)
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:488)
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:483)
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:483)
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:483)
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$400(TransformHierarchy.java:232)
	at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:207)
	at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:384)
	at org.apache.beam.runners.spark.translation.streaming.SparkRunnerStreamingContextFactory.call(SparkRunnerStreamingContextFactory.java:88)
	at org.apache.beam.runners.spark.translation.streaming.SparkRunnerStreamingContextFactory.call(SparkRunnerStreamingContextFactory.java:47)
	at org.apache.spark.streaming.api.java.JavaStreamingContext$$anonfun$10.apply(JavaStreamingContext.scala:776)
	at org.apache.spark.streaming.api.java.JavaStreamingContext$$anonfun$10.apply(JavaStreamingContext.scala:775)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.streaming.StreamingContext$.getOrCreate(StreamingContext.scala:864)
	at org.apache.spark.streaming.api.java.JavaStreamingContext$.getOrCreate(JavaStreamingContext.scala:775)
	at org.apache.spark.streaming.api.java.JavaStreamingContext.getOrCreate(JavaStreamingContext.scala)
	at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:155)
	at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:85)
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:277)
	at org.apache.beam.integration.nexmark.NexmarkRunner.run(NexmarkRunner.java:1281)
	at org.apache.beam.integration.nexmark.NexmarkDriver.runAll(NexmarkDriver.java:69)
	at org.apache.beam.integration.nexmark.drivers.NexmarkSparkDriver.main(NexmarkSparkDriver.java:46)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

https://issues.apache.org/jira/browse/BEAM-2112

@echauchot echauchot self-assigned this Apr 28, 2017
@iemejia iemejia changed the title query7: fails in spark in streaming mode, need support for PCollectionView in spark in streaming mode query7: fails in spark on streaming mode, need support for PCollectionView in spark in streaming mode Jun 9, 2017
@iemejia iemejia changed the title query7: fails in spark on streaming mode, need support for PCollectionView in spark in streaming mode query7: fails in spark on streaming mode, (No PCollectionView in streaming mode) Jun 9, 2017
@iemejia iemejia changed the title query7: fails in spark on streaming mode, (No PCollectionView in streaming mode) query7: fails in spark on streaming mode (not PCollectionView in streaming mode) Jun 9, 2017
iemejia pushed a commit that referenced this issue Jan 12, 2018
iemejia pushed a commit that referenced this issue Apr 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants