Predeploy pod throwing errors

[toland]

Quoting @bernardd:

[D]eployments are getting an error on the predeploy pod:

[INFO][2019-07-02 02:01:03 +0000] Deploying Pod/wocky-predeploy-a0ccef4e816ff74a (timeout: 600s)
[INFO][2019-07-02 02:01:05 +0000] Streaming logs from Pod/wocky-predeploy-a0ccef4e-816ff74a container 'wocky':
[INFO][2019-07-02 02:01:11 +0000] --logger-sasl-reports : Unknown option
[ERROR][2019-07-02 02:01:14 +0000] Pod/wocky-predeploy-a0ccef4e-816ff74a failed to deploy after 11.1s
[INFO][2019-07-02 02:01:14 +0000]
[INFO][2019-07-02 02:01:14 +0000] ------------------------------------------Result: FAILURE-------------------------------------------
[FATAL][2019-07-02 02:01:14 +0000] Failed to deploy 1 priority resource
[FATAL][2019-07-02 02:01:14 +0000]
[FATAL][2019-07-02 02:01:14 +0000] Pod/wocky-predeploy-a0ccef4e-816ff74a: FAILED
[FATAL][2019-07-02 02:01:14 +0000] Pod status: Failed.
[FATAL][2019-07-02 02:01:14 +0000]
[FATAL][2019-07-02 02:01:14 +0000] - Final status: Failed
[FATAL][2019-07-02 02:01:14 +0000] - Events: None found. Please check your usual logging service (e.g. Splunk).

>
> [H]owever it's not causing the deploy to register a failure, so there's really two issues there (the error itself and the fact that it's not causing the process as a whole to fail).

[toland]

[H]owever it's not causing the deploy to register a failure, so there's really two issues there (the error itself and the fact that it's not causing the process as a whole to fail).

The deploy failed, but the CI task succeeded. This is a known issue and probably requires a separate ticket. At first blush, it looks like kubernetes-deploy is not returning an error code when the deploy fails.

[toland]

[D]eployments are getting an error on the predeploy pod:
[...snip...]
--logger-sasl-reports : Unknown option

This was a bug in Distillery: bitwalker/distillery#687.

Distillery was updated via Dependabot today, and I merged the PR this morning. This shouldn't happen again.

[toland]

At first blush, it looks like kubernetes-deploy is not returning an error code when the deploy fails.

Actually, kubernetes-deploy is returning an error code, but other commands after it in the deployment script are obscuring the failure. It should be easy enough to fix.