Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 2048451: Fix proxy dial to pick all proxies #5743

Merged
merged 2 commits into from Apr 21, 2022

Conversation

rna-afk
Copy link
Contributor

@rna-afk rna-afk commented Mar 24, 2022

The current proxy dial function does not consider proxy
information provided in some environment variables like
http_proxy etc when checking if the service principal
provided is reachable. Adding a function that checks a few
env variables and dials the service to see if it is reachable.

@openshift-ci openshift-ci bot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Mar 24, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 24, 2022

@rna-afk: This pull request references Bugzilla bug 2048451, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.11.0) matches configured target release for branch (4.11.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (yunjiang@redhat.com), skipping review request.

In response to this:

Bug 2048451: Fix proxy dial to pick all proxies

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

if err == nil {
resp, err := http.DefaultClient.Do(req.Request)
if err == nil {
if resp.StatusCode == 200 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A GET to the endpoint itself will not necessarily result in a 200.

For example,

$ curl -I https://elasticloadbalancing.us-east-1.amazonaws.com
HTTP/1.1 404 Not Found

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping to change that to CONNECT and forgot to do that. :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ curl -I https://elasticloadbalancing.us-east-1.amazonaws.com -X CONNECT
HTTP/1.1 400 Bad Request

@patrickdillon
Copy link
Contributor

patrickdillon commented Mar 25, 2022

/hold
for discussion

See https://go.dev/doc/go1.16#net/http:

The ProxyFromEnvironment function no longer returns the setting of the HTTP_PROXY environment variable for https:// URLs when HTTPS_PROXY is unset.

Also golang/go#40909

It looks to me like the proxy environment could be misconfigured: HTTPS_PROXY needs to be set for requests to https://

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 25, 2022
@patrickdillon
Copy link
Contributor

Where did context for the HTTP_PROXY env variable in the description come from? I don't see it discussed in the bz...

@rna-afk
Copy link
Contributor Author

rna-afk commented Mar 25, 2022

I asked @yunjiang29 on how he tested it and he used the http_proxy and the https_proxy env variable to test the proxy connection. We still need to fix the existing code that only checks NO_PROXY and ALL_PROXY variables since proxy.Dial function does not check the http proxies.

@patrickdillon
Copy link
Contributor

I asked @yunjiang29 on how he tested it and he used the http_proxy and the https_proxy env variable to test the proxy connection. We still need to fix the existing code that only checks NO_PROXY and ALL_PROXY variables since proxy.Dial function does not check the http proxies.

Oh, I see.

/hold cancel

DefaultTransport respects these environment variables and is used by http.Client in the stdlib. Can we just that? It looks like we started off with net.Dial, which is perhaps a more specific use case than we need here.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 25, 2022
@patrickdillon
Copy link
Contributor

Oh, and I now see that is essentially what you are doing.

Copy link
Contributor

@patrickdillon patrickdillon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a moment to catch up. Perhaps I'm still catching up, but it seems like we can simplify this. If I'm missing edge cases, let's make them more explicit.

pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
@rna-afk rna-afk force-pushed the fix_proxy_dial_aws branch 3 times, most recently from 250dcdb to ad8c333 Compare March 29, 2022 12:38
@patrickdillon
Copy link
Contributor

/approve

Would like to get @staebler's sanity check.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 29, 2022
pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
pkg/asset/installconfig/aws/validation.go Outdated Show resolved Hide resolved
@sadasu
Copy link
Contributor

sadasu commented Apr 5, 2022

@rna-afk could we add unit tests to verify behavior when proxy env variables are set?

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 18, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: patrickdillon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

The current proxy dial function does not consider proxy
information provided in some environment variables like
http_proxy etc when checking if the service principal
provided is reachable. Adding a function that checks a few
env variables and dials the service to see if it is reachable.
Removing net packages as they are no longer being directly
used. Moving them to indirect packages.
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 19, 2022
Comment on lines +635 to +639
if test.proxy != "" {
os.Setenv("HTTP_PROXY", test.proxy)
} else {
os.Unsetenv("HTTP_PROXY")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it stands, I don't think test.proxy is worth including in these tests. We're not testing whether the proxy is being used or not. So I don't think including the proxy in the test is doing much.

If we were going to write a test for the proxy, I would imagine it would use something like [httpTest](https://pkg.go.dev/net/http/httptest#example-Server) and create a proxy server and make sure the proxy is being called when trying to connect to the endpoint.

While that could be cool, it seems like a fair amount of effort to test code that is mostly in the stdlib--not code we're writing. So I think it might be best to just remove these parts from the test.

@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 19, 2022
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 19, 2022
@patrickdillon
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 20, 2022
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

3 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 21, 2022

@rna-afk: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-e2e-aws-upgrade 96cddf2 link false /test okd-e2e-aws-upgrade
ci/prow/e2e-aws-upgrade 96cddf2 link true /test e2e-aws-upgrade
ci/prow/e2e-gcp-upgrade 96cddf2 link true /test e2e-gcp-upgrade
ci/prow/openstack-manifests 52e829b link true /test openstack-manifests
ci/prow/e2e-metal-ipi-ovn-ipv6 52e829b link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-aws-workers-rhel8 52e829b link false /test e2e-aws-workers-rhel8
ci/prow/e2e-aws-disruptive 52e829b link false /test e2e-aws-disruptive
ci/prow/e2e-aws-single-node 56f3264 link false /test e2e-aws-single-node
ci/prow/e2e-libvirt 56f3264 link false /test e2e-libvirt
ci/prow/e2e-crc 56f3264 link false /test e2e-crc
ci/prow/e2e-aws-fips 56f3264 link false /test e2e-aws-fips
ci/prow/e2e-aws-proxy 56f3264 link false /test e2e-aws-proxy
ci/prow/e2e-metal-assisted 56f3264 link false /test e2e-metal-assisted
ci/prow/e2e-ibmcloud 56f3264 link false /test e2e-ibmcloud
ci/prow/okd-e2e-aws 56f3264 link false /test okd-e2e-aws
ci/prow/e2e-ovirt 56f3264 link false /test e2e-ovirt
ci/prow/e2e-azure 56f3264 link false /test e2e-azure
ci/prow/e2e-vsphere 56f3264 link false /test e2e-vsphere
ci/prow/e2e-openstack 56f3264 link false /test e2e-openstack
ci/prow/e2e-aws-shared-vpc 56f3264 link false /test e2e-aws-shared-vpc
ci/prow/e2e-gcp 56f3264 link false /test e2e-gcp

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

8 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 09af68f into openshift:master Apr 21, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 21, 2022

@rna-afk: All pull requests linked via external trackers have merged:

Bugzilla bug 2048451 has been moved to the MODIFIED state.

In response to this:

Bug 2048451: Fix proxy dial to pick all proxies

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants