Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update go-getter client options #111

Merged
merged 5 commits into from Jun 9, 2022
Merged

Conversation

nywilken
Copy link
Member

@nywilken nywilken commented May 25, 2022

  • Disable reading and writing files through symlinks
  • Add timeouts for GCS and S3 getters

Related to: https://github.com/hashicorp/packer-internal-issues/issues/38 (internal only)

@nywilken nywilken requested a review from a team as a code owner May 25, 2022 01:21
@nywilken nywilken marked this pull request as draft May 25, 2022 01:22
@nywilken nywilken changed the title Update go-getter client options to disable symlinks Update go-getter client options May 25, 2022
@nywilken nywilken marked this pull request as ready for review May 27, 2022 14:56
Copy link
Member

@sylviamoss sylviamoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼
Since this one is a bigger change that might break existing templates, do you think we should add the changelog entry in this same PR with more information about the changes? Maybe mentioning the bump as well #110

DisableSymlinks: true,
// The order of the Getters in the list may affect the result
// depending if the Request.Src is detected as valid by multiple getters
Getters: []getter.Getter{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@nywilken nywilken force-pushed the nywilken/update-go-getter-client branch from faee09d to f0617fc Compare June 2, 2022 20:17
@nywilken nywilken requested a review from picatz June 2, 2022 20:37
Netrc: true,
XTerraformGetDisabled: true,
HeadFirstTimeout: 10 * time.Second,
ReadTimeout: 30 * time.Second,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other getter timeouts are 5 minutes, while this one is essentially 30 seconds. Maybe it'd make more sense to have a uniform timeout across the getters, or somehow make these configurable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of the uniform timeout. TBH 30 seconds seems really short given the use of go-getter in Packer. I struggled a bit to find good defaults. But ultimately think that going high 30 minutes is a good base line. We can make these configurable for the step so that plugins could override if needed.

That said there is an open issue for improving iso_config to support some addition go-getter Headers, which could also include setting some of these values. Taking a page out of Nomad's book here.

&getter.HgGetter{
Timeout: 5 * time.Minute,
},
new(getter.SmbClientGetter),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Should this getter have a timeout? I see this getter only exists in v2, and doesn't have that option currently. Just curious to your thoughts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - I think it should. It was not caught in the go-getter updates. I will follow up with a separate PR for that change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR opened for the SmbClientGetter hashicorp/go-getter#369

@nywilken nywilken force-pushed the nywilken/update-go-getter-client branch from f0617fc to 900cdcf Compare June 6, 2022 16:21
@nywilken nywilken merged commit 73f8475 into main Jun 9, 2022
@nywilken nywilken deleted the nywilken/update-go-getter-client branch June 9, 2022 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants