JSON serialization performance on Blazor

[steveharter]

Auguest 4, 2020
Steve Harter

Overview - serialization performance on Blazor

The 5.0 performance of System.Text.Json in serializer is slow under Blazor client but mostly consistent with other libraries, and is ~2x faster than 3.1. A rough expected multiplier is that the Mono interpreter should be at most 15x slower than the Mono JIT, and the serializer is in line with that in real-world scenarios.

It is believed the serializer and underlying reader\writer are not currently doing anything systematically that causes the serializer to be exceptionally slower under Blazor than other non-serializer scenarios. However, there are cases where focused improvements can be made including improving Span<T> performance.

Some super-slow areas were identified and recently fixed (or in progress).

The 15x multiplier compares the Mono interpreter against Mono JIT because that is the easiest way to measure progress (without trying to run Blazor each time). However, the community will compare Blazor against Core JIT and currently that multiplier is ~35-90x, at least for the main scenario covered here -- some scenarios will be faster or slower than that, but the 35-90x range should be a good approximation of a standard real-world scenario and what will likely be reported by the community.

For Blazor, this may or may not be performant enough depending on JSON payload size or object graph complexity. There have been earlier reports from the community that serialization is too slow although there have been improvements since then.

Blazor performance is poor, in general, because the IL is interpreted, not JITted or Crossgen'd. In addition, the WASM layer adds an additional 1.3-1.6x multiplier. Overall, these factors are known and are an expected tradeoff for the current architecture.

Improving Performance

Blazor performance can be improved in several ways; some example PRs and issues shown below:

• Adding intrinsics or optimizations to the interpreter for special cases.
  • interp Intrinsify span ctor
  • Optimize vector operations in mono interpreter
• Improving the interpreter perf in general such as adding better support for inlining.
  • Support AggressiveInlining in mono interpreter?
  • Add support for removing dead branches in mono interpreter
• Changing the source of various libraries to avoid certain calls or patterns.
  • Micro optimizations for serialization
  • Improve JSON (de)serialization perf of longer strings on mono
  • Improve polymorphic performance of (de)serialization
• Improving the WASM layer.

Primary real-world benchmark and observations

A real-world benchmark for a large object graph (339,803 characters) where serialize\deserialize is run 10 times:

test	Core JIT (40x warmup)	Core JIT (2x warmup)	Mono JIT	Mono interpreter	Blazor	Blazor vs Core 2x warmup	Blazor vs Core 40x warmup
Serialize	13ms	34	58	689	1195	35x slower	92x slower
Deserialize	53	106	138	2484	3724	35x slower	70x slower

The "Blazor vs Core" column uses both a 2x and 40x warmup factor. Although Blazor runs on the Mono interpreter, not Core JIT, the comparison is there since that will likely be what the community will compare against. The 2x warmup factor is likely more real-world than a 40x warmup for typical browser scenarios. The Core JIT, due to tiered JIT support, becomes faster after a few runs and from local tests about 40x runs are needed to get steady-state maximum performance.

Observations from this single benchmark:

• Blazor serialization for this scenario is ~35-90x slower than Core JIT (depending upon the warm-up factor). This is what the community will report for similar scenarios\benchmarks.
• Mono interpreter is 12-18x slower than Mono JIT. This is roughly in line with a 15x expectation based on other areas. The community is not likely to compare Blazor to Mono JIT.
• Mono interpreter is 1.5-1.7x slower when running under WASM.
• Since the benchmark runs 10 times, a single serialization took 1195\10 = 120ms and deserialization took 3724\10=372ms on Blazor. This seems fast enough if it is a single action responding to a UI event like a page open or button click, however running a phone or other slower device may not be acceptable. But this benchmark is on the large side (332K) so it all depends on the object graph and\or JSON size, and of course actual community feedback on whether real-world performance is acceptable.

5.0 work remaining

The serialization investigation work I performed to understand the current performance characteristics and potential areas to improve is complete. I am not aware of any more known low-hanging-fruit although there are cases the Mono team could intrinsify or improve.

One slow area is support for Span<T>. Since the Serializer (and underlying reader\writer) uses Span<T> in many places, some overall improvement should be gained. Here's a simple benchmark doing new Span<byte>(bytes).Slice() (100,000,000 iterations)

Core JIT	Mono JIT	Mono interpreter
23ms	652 (28x slower than Core)	7597 (330x slower than Core)

However, since there are so many iterations in the bechmark (100 million), significant perf gain is not expected (100 million operations takes 7.6 seconds). This issue is covered by interp Intrinsify span ctor.

Other areas:

• System.Text.Encodings.Web.DefaultJavaScriptEncoderBasicLatin:FindFirstCharacterToEncode although in that case there will likely need to be a fast-path (when default encoder is used) vs slow-path (non-default encoder).
• Improve Base64-encoding and decoding.
  • System.Buffers.Base64.DecodeFromUtf8
  • System.Text.Rune.EncodeToUtf8

Since these are non-trivial work items, there would need a cost:benefit for each and additional tests added to ensure quality.

Other activity from the Mono team members and suggestions:

• Vlad Brezae @BrzVlad and others continue to investigate and make interpreter changes for serialization as well as other areas.
• Additional research to improve general interpreter performance and\or to minimize the WASM overhead.
• Benchmarks comparing 3.1x against 5.0 especially if most of the "slow feedback" comes from 3.1x users. Currently this data is not readily available although it is expected that 5.0 is significantly faster. Besides the 5.0 Mono\Blazor improvements made by the Mono team members, the Serializer made several improvements in 5.0 that help under the interpreter.

Note it is possible that much of the interpreter work done in 5.0 is "throw-away" pending future plans (if the interpreter is not used).

Post-5.0 \ future

After 5.0, other layering options to improve performance will likely be considered that will avoid the IL interpreter overhead. This will likely be AOT-based such as compiling IL to native code, compiling IL to WASM bytecode or compiling IL to WASM binary.

There is a Json code-gen effort that will be ready in 6.0 that is expected to improve throughput performance 10-15% and minimize startup time of the first (de)serialization. It requires a code-gen plugin to generate the (de)serialization code for the POCOs in a given project.

Benchmarks

Existing JSON Benchmarks from https://github.com/dotnet/performance

These show the differences between Mono JIT vs. Mono interpreter.

There are about ~300 JSON benchmarks which range from 2.5x slower to 35x slower with the median ~11x slower.

The benchmarks that are in the 20x+ range are mostly micro-benchmarks of the reader and writer or very small POCOs (which will be super quick) and not general real-world (de)serialization scenarios of larger object graphs or collections. However, there are some Base64 benchmarks that fall into this 20x+ range can be improved if we intrinsify that; Base64 is used when a property is byte[].

Consider the benchmark System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes which is a real-world object model and runs ~12x slower. The corresponding System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes runs ~8x slower. These numbers should get a bit faster with Improve JSON (de)serialization perf of longer strings on mono.

Click to expand IndexViewModel object model

    public class IndexViewModel 
    {
        public List<ActiveOrUpcomingEvent> ActiveOrUpcomingEvents { get; set; } // list of 20 items
        public CampaignSummaryViewModel FeaturedCampaign { get; set; }
        public bool IsNewAccount { get; set; }
    }

    public class ActiveOrUpcomingEvent
    {
        public int Id { get; set; }
        public string ImageUrl { get; set; }
        public string Name { get; set; }
        public string CampaignName { get; set; }
        public string CampaignManagedOrganizerName { get; set; }
        public string Description { get; set; }
        public DateTimeOffset StartDate { get; set; }
        public DateTimeOffset EndDate { get; set; }
    }

    public class CampaignSummaryViewModel
    {
        public int Id { get; set; }
        public string Title { get; set; }
        public string Description { get; set; }
        public string ImageUrl { get; set; }
        public string OrganizationName { get; set; }
        public string Headline { get; set; }
    }    

Instructions to view benchmarks:

• https://aka.ms/dotnetperfindex
• Navigate:
  • Daily ->
  • dotnetruntime ->
  • refs-heads-master ->
  • x64 ->
  • Ubuntu1804 ->
  • CompilationMode:tiered ->
  • MonoInterpreter:false + MonoInterpreter:true ->
  • JSON

WASM

WASM (de)serialization performance from the first release of Blazor (3.1x) until has increased by around 2x. See page 14 at
https://msit.powerbi.com/view?r=eyJrIjoiYTZjMTk3YjEtMzQ3Yi00NTI5LTg5ZDItNmUyMGRlOTkwMGRlIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9
or this image:

todo: from the graph, determine when the switch to 5.0 runtime version of System.Text.Json.dll occurred. There have been several System.Text.Json perf improvements in 5.0.

Large object graph benchmark

This benchmark is used in the Primary real-world benchmark and observations section above.

Click to expand

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text.Json;

namespace Benchmark
{
    class Program
    {
        static void Main(string[] args)
        {
            Benchmark(out string serialize, out string deserialize);

            Console.WriteLine($"{serialize}");
            Console.WriteLine($"{deserialize}");
        }

        public static void Benchmark(out string serialize, out string deserialize)
        {
            string json = "";

            var largeOrgChart = Person.GenerateOrgChart(5, 4);

            // Ensure all caches are warmed up
            // A factor of 40 instead of 2 can be used to maximize Core JIT performance.
            const int WarmUpIterations = 2;
            for (int i = 0; i < WarmUpIterations; i++)
            {
                json = JsonSerializer.Serialize(largeOrgChart);
                JsonSerializer.Deserialize<Person>(json);
            }

            Stopwatch stopWatch = new Stopwatch();
            stopWatch.Start();
            for (int i = 0; i < 10; i++)
            {
                json = JsonSerializer.Serialize(largeOrgChart);
            }
            stopWatch.Stop();
            serialize = $"Serialization took {stopWatch.ElapsedMilliseconds} ms";

            stopWatch.Reset();
            stopWatch.Start();
            Person person;
            for (int i = 0; i < 10; i++)
            {
                person = JsonSerializer.Deserialize<Person>(json);
            }
            stopWatch.Stop();
            deserialize = $"Deserialization took {stopWatch.ElapsedMilliseconds} ms";
        }

        public class Person
        {
            static readonly string[] Clearances = new[] { "Alpha", "Beta", "Gamma", "Delta", "Epsilon" };

            public string Name { get; set; }
            public int Salary { get; set; }
            public bool IsAdmin { get; set; }
            public List<Person> Subordinates { get; set; }

            // Note: this uses System.Object as the dictionary value which enters a slow polymorphic mode
            // during deserialization since it creates a JsonElement. A System.Boolean instead (since it
            // it will always be a bool in this benchmark) will increase deserialization performance significantly.
            public Dictionary<string, object> SecurityClearances { get; set; }

            public static Person GenerateOrgChart(int totalDepth, int numDescendantsPerNode, int thisDepth = 0, string namePrefix = null, int siblingIndex = 0)
            {
                var name = $"{namePrefix ?? "CEO"} - Subordinate {siblingIndex}";
                var rng = new Random(0);
                return new Person
                {
                    Name = name,
                    IsAdmin = siblingIndex % 2 == 0,
                    Salary = 10000000 / (thisDepth + 1),
                    SecurityClearances = Clearances
                        .ToDictionary(c => c, _ => (object)(rng.Next(0, 2) == 0)),
                    Subordinates = Enumerable.Range(0, thisDepth < totalDepth ? numDescendantsPerNode : 0)
                        .Select(index => GenerateOrgChart(totalDepth, numDescendantsPerNode, thisDepth + 1, name, index))
                        .ToList()
                };
            }
        }
    }
}

Small POCO

In order to measure string-size performance, a small POCO was used. This was also used in #39733.

    public class SimpleStructWithProperties
    {
        public int Num { get; set; }
        public string Text { get; set; }
    }

String size of Text property ranged from 10 characters to 10,240 characters.

Serializer details

Some notes on the serializer that highlights the main areas of work.

The serializer uses the Utf8JsonWriter and deserializer uses the Utf8JsonReader. The majority of time is spent in the reader\writer, not the (de)serializer.

Some areas that are hot spots for past performance work:

• Deserialization
  • Unescaping.
  • Property lookup (matching JSON property name to CLR property).
  • Calling the constructor (IL Emit used if available).
  • Calling property setters (IL Emit used if available).
  • (Re)allocating buffer(s) for the reader.
  • Transcoding UTF8 strings to UTF16.
• Serialization
  • Escaping values.
  • Escaping property names (cached).
  • Calling property getters (IL Emit used if available).
  • Enumerating collections (optimized by using specific converters to avoid boxing).
  • (Re)allocating buffer(s) for the writer.
  • Transcoding UTF16 strings to UTF8.

Misc notes:

• The reader\writer make heavy use of Span<T> which is very slow under both Mono JIT and Mono interpreter. Common scenarios including new Span<T> and Slice().
• System.Text.Json makes heavy use of AggressiveInlining. The interpreter does not support the attribute; covered by Support AggressiveInlining in mono interpreter? #39632.
• Encoding and escaping use the System.Text.Encodings.Web assembly and the serializer allows a pluggable encoder that can be specified on JsonSerializerOptions. Because the encoder is pluggable, it makes it more difficult to create interpreter intrinstics for that.
• The Async serialization methods use a Stream and are ~10% slower than non-async that use byte[] or string. However, the async mode supports a true streaming mode: during deserialization, no "drain" is done upfront during deserialization and the deserializer will continue to ask the Stream for more data; during serialization FlushAsync() is called automatically to minimize buffer size and memory consumption. Thus the Async mode may perform better than sync mode in large payload + memory-exhausted scenarios or when the Stream is itself is reading or writing over a higher-latency endpoint or physical file since the (d)serializer can start the work before waiting for the whole Stream to load.

[szalapski]

Thanks for looking into this. So this deserialization slowness affects nearly every Blazor WebAssembly app, everywhere, right? Small requests and large requests alike will take 35 to 90 times longer than expected, for every single HTTP request that does a JSON deserialization? I am surprised that this has received so little attention. Isn't this a huge drag on everyone's performance everywhere?

Also, any ideas for working around--is the only way to make responses smaller?

[steveharter]

@szalapski are you currently using WebAssembly against the 3.1 libraries, and is (de)serialization fast enough for your scenarios?

If the serialization scenarios are replying to client UI events and have a "typical" payload size and complexity, I think performance should be fine (i.e. not affecting UI responsiveness) especially since Blazor serialization has improved in 5.0 by around 2x.

Small requests and large requests alike will take 35 to 90 times longer than expected, for every single HTTP request that does a JSON deserialization

Yes the 35x-90x factor is a conservative number for "real-world" scenarios. The 90x high end is compared against a fully warmed-up (tiered compilation) Core JIT as well, which may not be a fair comparison since Blazor client scenarios will likely not run enough times to kick in the tiered compilation (if that same serialization code was run against the Core JIT). Thus the lower range at around 35x range may be a more fair comparison IMO.

[szalapski]

Thanks for responding, @steveharter. I have a real-world app where it takes 10-30 seconds to deserialize 1-6 MB of data, and this is not fast enough for us. Does this fit with what you say? #40386 I still have a hard time believing "it's just slow" here. Something seems wrong.

I am using Blazor 3.2.0 with System.Text.Json 5.0.0-preview.7.

[steveharter]

Thanks I replied in #40386