Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[jvm-packages] cancel job instead of killing SparkContext #6019

Merged
merged 5 commits into from Sep 2, 2020
Merged

[jvm-packages] cancel job instead of killing SparkContext #6019

merged 5 commits into from Sep 2, 2020

Commits on Aug 26, 2020

  1. cancel job instead of killing SparkContext 

    This PR changes the default behavior that kills SparkContext. Instead, This PR
cancels jobs when coming across task failed. That means the SparkContext is
still alive even some exceptions happen.
    @wbo4958
    wbo4958 committed Aug 26, 2020
    Copy the full SHA
    2a2b47a View commit details
    Browse the repository at this point in the history

  2. add a parameter to control if killing SparkContext

    @wbo4958
    wbo4958 committed Aug 26, 2020
    Copy the full SHA
    f834537 View commit details
    Browse the repository at this point in the history

  3. cancel the jobs the failed task belongs to

    @wbo4958
    wbo4958 committed Aug 26, 2020
    Copy the full SHA
    8852f59 View commit details
    Browse the repository at this point in the history

  4. remove the jobId from the map when one job failed.

    @wbo4958
    wbo4958 committed Aug 26, 2020
    Copy the full SHA
    f1df786 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2020

  1. resolve comments

    @wbo4958
    wbo4958 committed Aug 31, 2020
    Copy the full SHA
    2824d66 View commit details
    Browse the repository at this point in the history