Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes: add retry to apply original.yaml after crd is installed #4066

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zhongweili2010
Copy link

What this PR does / why we need it:

To accommodate slower machines when apply original installer spec, install spec sometimes fails after crd is already installed.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Steps to reproduce

Does this PR introduce a user-facing change?

NONE

Does this PR require documentation?

NONE

@zhongweili2010 zhongweili2010 requested a review from a team as a code owner February 9, 2023 22:15
@emosbaugh
Copy link
Member

Hello. We appreciate the contribution! Would it be possible to attach logs here from the output of the installation script reproducing the issue to help me better understand? Thank you!

@@ -125,7 +125,7 @@ function apply_installer_crd() {
cat > $ORIGINAL_INSTALLER_SPEC <<EOL
${INSTALLER_YAML}
EOL
kubectl apply -f "$ORIGINAL_INSTALLER_SPEC"
try_1m kubectl apply -f "$ORIGINAL_INSTALLER_SPEC"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The line just apply the manifest.
The try will be re-doing the same command each 2 seconds :

# try a command every 2 seconds until it succeeds, up to 30 tries max; useful for kubectl commands
# where the Kubernetes API could be restarting
function try_1m() {
local fn="$1"
local args=${@:2}
n=0
while ! $fn $args 2>/dev/null ; do
n="$(( $n + 1 ))"
if [ "$n" -ge "30" ]; then
# for the final try print the error and let it exit
echo ""
try_output="$($fn $args 2>&1)" || true
echo "$try_output"
bail "spent 1m attempting to run \"$fn $args\" without success"
fi
sleep 2
done
}

So, why would need to be required to re-duo kubectl apply many times?
What would be solved with that?

+1 to the @emosbaugh comments. That would be great if you could share the problem, and all logs of the issue that you are facing so we can better check it out and see how we can help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants