Discrepancy in Serialized Size with MessagePack for Complex Object Graphs

[AbdulrahmanXSO25]

I'm encountering an issue where serializing a complex object graph using MessagePack results in an unexpectedly small serialized size compared to directly serializing a collection of documents contained within the object. This discrepancy suggests that the documents are not being serialized as part of the complex object graph as expected.

Serializing a complex object that includes a collection of documents should reflect the total size of the serialized documents when serialized together as part of the complex object.

The serialized size of the complex object is significantly smaller than the size of the directly serialized collection of documents, suggesting that the collection is not being serialized as expected.

Steps to Reproduce:

1. Define a Page class with a Dictionary<string, Document> property, annotated with [MessagePackObject].
2. Populate the dictionary with several Document instances.
3. Serialize the Page object using MessagePackSerializer.Serialize.
4. Compare the serialized size to the size of the documents serialized directly.

Environment:

• MessagePack version:
• .NET version: 8.0
• Operating System: Windows

I've already tried ensuring that my classes are correctly annotated for MessagePack serialization and explicitly specifying serialization options, but the issue persists.

Any insights or suggestions on what might be causing this behavior would be greatly appreciated.

using MessagePack;
using MessagePack.Resolvers;
using System.Diagnostics;

class Program
{
    static void Main(string[] args)
    {
        var x = 5;
        // Inserting x fake documents
        var documents = ProgramHelpers.GenerateXRows(x);
        var page = new Page(0);
        foreach (var document in documents)
        {
            page.AddDocument(document);
        }

        var pageSerialized = DebugSerializeToJson(page);

        var documentsSerialized = DebugSerializeToJson(documents);

        Console.WriteLine(pageSerialized.Length);
        Console.WriteLine(documentsSerialized.Length);
    }

    public static string DebugSerializeToJson<T>(T obj)
    {
        var options = MessagePackSerializerOptions.Standard.WithResolver(ContractlessStandardResolver.Instance);
        byte[] bytes = MessagePackSerializer.Serialize(obj, options);
        string json = MessagePackSerializer.ConvertToJson(bytes);
        return json;
    }
}

Page.cs:
using MessagePack;

[MessagePackObject(keyAsPropertyName: true)]
public class Page
{
    [IgnoreMember]
    private const int PageSize = 8192; // 8KB
    public int PageNumber { get; }
    public Dictionary<string, Document> Documents { get; }

    public Page(int pageNumber)
    {
        PageNumber = pageNumber;
        Documents = new Dictionary<string, Document>();
    }
    
    public void AddDocument(Document document)
    {
        Documents.Add(document.Id, document);
    }
}

using MessagePack;

[MessagePackObject(keyAsPropertyName: true)]
public class Document
{
    public string Id { get; set; }
    public Dictionary<string, object> Data { get; }

    public Document(Dictionary<string, object> data)
    {
        Id = UniqueIdentifierGenerator.GenerateUniqueId();
        data["Id"] = Id;
        Data = data;
    }
}

[AArnott]

Thank you for the repro. I haven't time to investigate now, but one tip you could use is to run the resulting msgpack binary through our msgpack-to-json API so that you can see what actually got serialized and debug from there.