Sending mail with attachments ends in error "A task was canceled."

[vko-insiders]

Describe the bug

I am trying to send a mail with attachment on customer site but it fails with an exception of type System.Threading.Tasks.TaskCanceledException with message A task was canceled. (no inner message).
We are using Microsoft.Graph in version 4.54.0 with .NET 4.8 on Windows Server. Upgrading to a v5 is planned but not possible at the moment.

There are some curious things about it:
Sending mails with and without attachment works in our environment.
Sending mails without attachment is possible on customer site.
Sending mails with attachment fails with said exception.

Trying to send mails with attachment via PowerShell is possible on customer site.
Customer uses Proxy and Firewalls.
We are utilizing the proxy and can rule out Firewall issues because when calling endpoints via browser like Chrome or Edge we get http error message 401 not authorized instead of 404 Not found, when we change the URI a bit.
We are logging requests and responses, but are not able to log session upload requests/responses (neither on customer site nor local site).
We can rule out the proxy as we also configured the proxy on Windows via Windows network settings and all other operations like fetching mails and sending mails without attachments work fine with that setting as well.

Expected behavior

I would expect to send mails with attachments on customer site.

How to reproduce

Create instance of GraphServiceClient where authProvider is a Delegated User Authentication, webProxy is our proxy and loggingHandler is logging the requests and responses (see below).

var httpClient = GraphClientFactory.Create(authProvider, proxy: webProxy, finalHandler: loggingHandler);
Client = new GraphServiceClient(httpClient);

Our logging handler is pretty simple

public class GraphLoggingMessageHandler : DelegatingHandler
{
    public GraphLoggingMessageHandler()
    {
        InnerHandler = new HttpClientHandler();
    }

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        request.Version = HttpVersion.Version11;
        HttpResponseMessage response = await GraphExceptionHandler.Call(async () =>
        {
            return await base.SendAsync(request, cancellationToken);
        });
        Log.Message("GraphLogging", await createLogMessage(request, response));
        return response;
    }

    private static async Task<string> createLogMessage(HttpRequestMessage request, HttpResponseMessage response)
    {
        string message = "=== Request ===\n" + request.ToString() + "\n";
        if (request.Content != null)
        {
            message += "Content: " + await request.Content.ReadAsStringAsync() + "\n";
        }

        message += "\n\n=== Response ===\n" + response.ToString() + "\n";
        if (response.Content != null)
        {
            message += "Content: " + await response.Content.ReadAsStringAsync() + "\n";
        }
        return message;
    }
}

Now the setup for sending mail with attachment is pretty much standard as well. We can see that a mail is created in Draft-folder and we can add attachments on our site and send it as expected, but adding attachments on customer site is not possible.

private async Task AddFileAttachments(Message draftMail, string[] attachmentPaths)
{
	foreach (var attachmentPath in attachmentPaths)
	{
		var attachmentContentSize = new FileInfo(attachmentPath).Length;
		var attachmentItem = new AttachmentItem
		{
			AttachmentType = AttachmentType.File,
			Name = Path.GetFileName(attachmentPath),
			Size = attachmentContentSize,
		};

		var uploadSession = await await Connection.Client.Users[Connection.User].Messages[draftMail.Id].Attachments
																.CreateUploadSession(attachmentItem)
																.Request()
																.PostAsync();
		var maxChunkSize = 1024 * 320;

		using (var filestream = System.IO.File.Open(attachmentPath, FileMode.Open, FileAccess.Read))
		{
			var streamLength = filestream.Length;
			filestream.Seek(0, SeekOrigin.Begin);

			LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize);

			IProgress<long> progress = new Progress<long>(prog =>
			{
				Log.Message($"ploaded {prog} bytes of {streamLength} bytes");
			});

			try
			{
				var result = await largeFileUploadTask.UploadAsync(progress);
			}
			catch (ServiceException ex)
			{
			}
			catch (Exception ex)
			{
			}
		}
	}
}

We also see no logging requests/responses done by largeFileUploadTask.
So we cannot share request-ID on that particular calls with Microsoft Support.

SDK Version

4.54.0

Latest version known to work for scenario above?

No response

Known Workarounds

No response

Debug output

No response

Configuration

Windows Server 2019, 64-bit, Version 1809, OS Build 17763.5696.

Other information

I suspect the issue to be somewhere around this:

LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize);
var result = await largeFileUploadTask.UploadAsync(); // here

[andrueastman]

Thanks for raising this @vko-insiders

As you are behind a proxy, the LargeFileUploadTask will create a client that does not respect the proxy configuration.

In the line below, the task should be able to take in a graphClient as an optional parameter. Could you try passing Connection.Client as the last parameter and confirm if this resolves the issue for you?

LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize);

[vko-insiders]

When I change

LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize);

to

LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize, Connection.Client);

I get a ServiceException saying
Microsoft.Graph.ServiceException: 'Code: generalException Message: An error occurred sending the request.'
with Inner Exception
InvalidOperationException: The stream was already consumed. It cannot be read again.

I also tried creating a new BaseClient insted but with the same result.
Removing Connection.Client leads to no such error message.

Did I do something wrong or did I miss something?

[vko-insiders]

Update: The same code on customer site throws a different error message of type ServiceException with

• Statuscode 0
• Message: Code: generalException\nMessage: An error occurred sending the request.
• Inner Exception Message An error occurred while sending the request..

[andrueastman]

I get a ServiceException saying
Microsoft.Graph.ServiceException: 'Code: generalException Message: An error occurred sending the request.'
with Inner Exception
InvalidOperationException: The stream was already consumed. It cannot be read again.

Does the error occur on the before the upload begins? Or during the upload?

I also tried creating a new BaseClient insted but with the same result.
Removing Connection.Client leads to no such error message.

Does this mean that the upload works out or do you still get the TaskCanceledException?

Statuscode 0 (generalException)

This suggests that the client was still unable to make a connection to the server. Any chance you're able to capture a trace with Fiddler to confirm?

[vko-insiders]

@andrueastman I was just able to fix local issue with InvalidOperationException: The stream was already consumed. It cannot be read again..
This was caused by GraphLoggingMessageHandler trying to read the stream again in request (see code in first post)

message += "Content: " + await request.Content.ReadAsStringAsync() + "\n";

I was able to fix this by exclude logging the content if the following is given request.Content is StreamContent.

Now I locally get the following error instead (with Connection.Client set):

• Type: ServiceException
• StatusCode: 0
• Message: Code: generalException\nMessage: An error occurred sending the request.
• Inner Message: Wert kann nicht hinzugefügt werden, da der Header "Authorization" nicht mehrere Werte unterstützt. which translates to Cannot add value because header 'Authorization' does not support multiple values

[vko-insiders]

I have an update: On customer site I get the following error with

LargeFileUploadTask<FileAttachment> largeFileUploadTask = new LargeFileUploadTask<FileAttachment>(uploadSession, filestream, maxChunkSize, Connection.Client);

The response reads

• StatusCode: 401
• ReasonPhrase Unauthorized
• Content: "error":{"code":"InvalidAudience","message":"The audience claim value is invalid '00000003-0000-0000-c000-000000000000/@'.","innerError":{"oAuthEventOperationId":"0124a91b-f31f-4211-9cd9-7ee65cc0b163","oAuthEventcV":"PKvuWWdU1kXDjt9VbY+m+Q.1.1","errorUrl":"https://aka.ms/autherrors#error-InvalidResource","requestId":"59eeab3c-5467-45d6-c38e-df556d8fa6f9","date":"2024-05-03T11:33:57"}}}

The full response log.

=== Response ===\nStatusCode: 401, ReasonPhrase: 'Unauthorized', Version: 1.1, Content: System.Net.Http.NoWriteNoSeekStreamContent, Headers: \n{ \n Cache-Control: private \n Date: Fri, 03 May 2024 11:33:56 GMT \n Transfer-Encoding: chunked \n Server: Microsoft-HTTPAPI/2.0 \n WWW-Authenticate: Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@*", token_types="app_asserted_user_v1 service_asserted_app_v1", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", error="invalid_token" \n X-BEServer: DU0PR02MB8191 \n X-NanoProxy: 1,1 \n X-AspNet-Version: 4.0.30319 \n Request-Id: 59eeab3c-5467-45d6-c38e-df556d8fa6f9 \n X-CalculatedFETarget: DUZPR01CU011.internal.outlook.com \n Alt-Svc: h3=":443";ma=2592000,h3-29=":443";ma=2592000 \n MS-CV: PKvuWWdU1kXDjt9VbY+m+Q.1.1 \n X-BackEndHttpStatus: 401,401 \n X-BeSku: WCS7 \n X-CalculatedBETarget: DU0PR02MB8191.eurprd02.prod.outlook.com \n X-DiagInfo: DU0PR02MB8191 \n X-FEEFZInfo: DUB \n X-UserType: Business \n X-FEProxyInfo: DUZPR01CA0201 \n X-FEServer: FR3P281CA0092 \n X-InternalRequestTiming: API:0,5 \n x-ms-diagnostics: 2000003;reason="The audience claim value is invalid '00000003-0000-0000-c000-000000000000/@'.";error_category="invalid_resource" \n X-Proxy-BackendServerStatus: 401 \n X-Proxy-RoutingCorrectness: 1 \n X-RUM-NotUpdateQueriedPath: 1 \n X-RUM-NotUpdateQueriedDbCopy: 1 \n X-RUM-Validated: 1 \n X-FirstHopCafeEFZ: HHN \n Strict-Transport-Security: max-age=31536000; includeSubDomains \n Content-Type: text/html; charset=utf-8 \n}\nContent: {"error":{"code":"InvalidAudience","message":"The audience claim value is invalid '00000003-0000-0000-c000-000000000000/@'.","innerError":{"oAuthEventOperationId":"0124a91b-f31f-4211-9cd9-7ee65cc0b163","oAuthEventcV":"PKvuWWdU1kXDjt9VbY+m+Q.1.1","errorUrl":"https://aka.ms/autherrors#error-InvalidResource","requestId":"59eeab3c-5467-45d6-c38e-df556d8fa6f9","date":"2024-05-03T11:33:57"}}}\n