CPU hotpath originating in CosmosQueryClientCore.GetCosmosElementResponse

[olivertowers]

Describe the bug
Seeing CPU hotpath that is originating in CosmosQueryClientCore.GetCosmosElementResponse which is using 6% of application CPU during a query workload.

Flame graph reference internal link:

Zooming into the QueryPage ctor:

The main issue is whether this CPU impact can be reduced when SDK is handling high volume query workloads.

Within this hotpath two methods are of note:

1. Page ctor is creating an ImmutableDictionary for AdditionalHeaders property, see - ImmutableDictionary is known to be much slower than Dictionary, so maybe IReadOnlyDictionary would be better for this even if the trade-off is that true immutability guarantee is lost?

2. QueryExceutionInfo header is deserialized with Newtonsoft.Json here - Would it be more performant to use the Json text readers built-in to the SDK instead?

To Reproduce

Run a high volume query workload and monitor/profile on the SDK.
More likely to see the issue with the following setup/workload

1. 10+ partitions collection.
2. Query workload issuing cross partition queries which return no results. This highlights the impact of many empty pages being returned but where the header/execution info is still significant.

Expected behavior
Expect a lower CPU impact on an application for the critical path in query responses.

Actual behavior
Higher CPU impact for query response handling especially when very few items are returned.

Environment summary
SDK Version: 3.21/Any
OS Version: Win 10

[kirankumarkolli]

@sourabh1007 what else is pending for this issue?

[sourabh1007]

It can be closed now.