ISSUE-9507: ADD application/gzip,application/octet-stream accept header

[mattclegg]

What this PR does:
This PR adds the accept header to the request sent by the HTTPGetter when downloading archives.

The function accepts gzip and so it would be preferable if it also notified the server about this. Additionally, this adds support for private helm charts hosted with GitHub.

Special notes for your reviewer:
Issue related to this are #9507

[robertsirc]

So, I will ask the question where are the test for this?

[mattclegg]

OK, @robertsirc
What would you like to see as a test for this?

• NewHTTPGetter to contain the accept Header
• Server with varied Accept Header
• Ensure headers set return related response

or did you have something else in mind?

You may also test with a private repository on GitHub via;

curl -L \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/$OWNER/$REPO/releases/assets/$ASSET_ID

which returns;

...
< content-type: application/json; charset=utf-8
...

but

curl -L \
  -H "Accept: application/octet-stream" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/$OWNER/$REPO/releases/assets/$ASSET_ID

downloads the asset.

[robertsirc]

Like in httpgetter_test we will expect the useragent and if it isn't correct it throws an error, for example in the test there called TestDownload I am questioning if we should have test to ensure a response if we are looking for it in the code. to remain idomatic of the rest of the codebase.

[mattclegg]

Thanks for clarifying @robertsirc
Like this; https://github.com/mattclegg/helm/pull/2/files ?

[robertsirc]

@mattclegg yes is this in one PR or two?

[mattclegg]

@robertsirc The other PRs are to avoid having noise on this branch for other tests that aren't needed.

I've merged the test for the server with varied Accept Header to this branch. Please let me know if there's anything else that needs to be done before this PR can be merged.