Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network retry on IOException (new in Jib 3.1.3) significantly slows down insecure registry failover (allowInsecureRegistries=true) #3422

Closed
chanseokoh opened this issue Aug 13, 2021 · 1 comment · Fixed by #3424

Comments

@chanseokoh
Copy link
Member

chanseokoh commented Aug 13, 2021

Problem identified in #3409 (comment).

We implemented retrying in #3351 (and a clean-up #3390) but overlooked the issue when allowInsecureRegistries=true, where insecure connection failover depends on SSLException (a subclass of IOException). Consequently, HTTP failover can happen only after exhausting retrying.

Base image 'amazoncorretto:16' does not use a specific image digest - build may not be reproducible
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will NOT be retried
Cannot verify server at https://localhost:5000/v2/. Attempting again with no TLS verification.
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will be retried
GET https://localhost:5000/v2/ failed and will NOT be retried
Failed to connect to https://localhost:5000/v2/ over HTTPS. Attempting again with HTTP.
@chanseokoh
Copy link
Member Author

What a bummer. My idea was to disable retrying if SSLException (almost all the time not a retryable situation), but not possible due to googleapis/google-http-java-client#251.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment