sporadic sourcelink test http request errors

[ryangribble]

On TravisCI OSX image builds, have been getting An error occurred while sending the request. error when running sourcelink test against the nupkg

Windows (AppVeyor) and Linux (TravisCI) builds do not seem to be affected, however about 50% of OSX builds seem to be running into this.

Is there any way to get more debug output, for example a stack trace or the Inner Exception (presumably from the http request being made to github to get the hash)?

The command being run by our Cake build is essentially dotnet sourcelink test <path to nupkg>

The task output:

========================================
TestSourceLink
========================================
Executing task: TestSourceLink
Testing sourcelink info in packaging/Octokit.0.30.0-PullRequest1738.nupkg
An error occurred while sending the request.
An error occurred when executing task 'TestSourceLink'.
Error: Sourcelink test failed!

I can't provide TravisCI logs because when you tell it to retry, it doesnt seem possible to get the earlier log anymore. But so far the OSX build will fail initially, then succeeded on the first retry... so 50%

[ryangribble]

Hmm just encountered a build where it worked on OSX but got this error on the Linux config. So I guess it's sporadically occurring on either

[ctaggart]

Probably just HTTP errors communicating with GitHub. It is downloading all of the actual source files and verifying that their checksums work.
https://github.com/ctaggart/SourceLink/blob/master/dotnet-sourcelink/Program.cs#L545

[ryangribble]

strange that we never seemed to encounter it prior to moving to the SDK2 tooling though?

I take it there is no way to see the InnerException apart from modifying sourcelink itself to catch exceptions and output additional information?

[ctaggart]

The good news is the source code isn't complicated and you can use the AppVeyor NuGet feed or I can push your PR builds to the MyGet SourceLink feed. Let me know.

[ryangribble]

Cool will have a look on the weekend, it would be nice if we can access the verbosity parameter of dotnet cmd and respond accordingly

[ctaggart]

Still an issue?

[ctaggart]

Reopen if it is. May be try dotnet 2.1?

[ryangribble]

Yeah we do still get a large percentage of failures on travis, I've not had time to look at adding anything into sourcelink to help troubleshoot or retry though sorry

May try 2.1 but my feeling is it's a networking issue from the travis container to github, whether it's abuse detection on github side or something on travis side I'm not sure

[ryangribble]

(Happy to leave this closed though since I'm not actively looking at this right now)

[ctaggart]

Reopening because I've been this error a lot. I'm one a fast internet connection. I have dotnet/sdk/2.1.300 installed and this is a netcoreapp2.1.

sourcelink test "C:\Users\taggac\Downloads\Newtonsoft.Json.11.0.3-beta1.nupkg"
sourcelink test passed: lib/net20/Newtonsoft.Json.pdb
sourcelink test passed: lib/net35/Newtonsoft.Json.pdb
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

[ryangribble]

So that's running locally yourself? Interesting...

Our travisCI builds have been (anecdotally) better recently, I haven't had this happen for several sequential builds whereas previously it was every couple of builds.

We are still netcoreapp2.0 but SDK is now latest (2.1.300)

[ctaggart]

I had a VPN on. After turning it off, The Newtonsoft.Json*.nupkg passed fives times in a row.

[ctaggart]

😱 Only failing when I don't have Fiddler enabled

PS C:\tmp>
PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
sourcelink test passed: lib/net20/Newtonsoft.Json.pdb
sourcelink test passed: lib/net35/Newtonsoft.Json.pdb
sourcelink test passed: lib/net40/Newtonsoft.Json.pdb
sourcelink test passed: lib/net45/Newtonsoft.Json.pdb
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
sourcelink test passed: lib/net20/Newtonsoft.Json.pdb
sourcelink test passed: lib/net35/Newtonsoft.Json.pdb
sourcelink test passed: lib/net40/Newtonsoft.Json.pdb
sourcelink test passed: lib/net45/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard1.0/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard1.3/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard2.0/Newtonsoft.Json.pdb
sourcelink test passed: lib/portable-net40+sl5+win8+wp8+wpa81/Newtonsoft.Json.pdb
sourcelink test passed: lib/portable-net45+win8+wp8+wpa81/Newtonsoft.Json.pdb
PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
sourcelink test passed: lib/net20/Newtonsoft.Json.pdb
sourcelink test passed: lib/net35/Newtonsoft.Json.pdb
sourcelink test passed: lib/net40/Newtonsoft.Json.pdb
sourcelink test passed: lib/net45/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard1.0/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard1.3/Newtonsoft.Json.pdb
sourcelink test passed: lib/netstandard2.0/Newtonsoft.Json.pdb
sourcelink test passed: lib/portable-net40+sl5+win8+wp8+wpa81/Newtonsoft.Json.pdb
sourcelink test passed: lib/portable-net45+win8+wp8+wpa81/Newtonsoft.Json.pdb
PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
sourcelink test passed: lib/net20/Newtonsoft.Json.pdb
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
PS C:\tmp>

[ctaggart]

I'm looking at the .NET Core 2.1.1 update and hoping it is the concurrent socket disposal bug. Installing the update...

[ctaggart]

That didn't help. If I set it to the old code with
https://github.com/dotnet/core/blob/master/release-notes/2.1/2.1.0.md#networking-performance

$env:DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER="false"

I still get network problems, but with a different error:

PS C:\tmp> sourcelink.exe test .\Newtonsoft.Json.11.0.3-beta1.nupkg
An error occurred while sending the request.

[ctaggart]

May be worth reviewing:
http://www.nimaara.com/2016/11/01/beware-of-the-net-httpclient/