16MB stack needed for 13kb JSON

[broccolihighkicks]

Hello,

I have quite a large graph of structs/enums, with the enums using untagged.

The graph has recursive types by using Box.

In general my types are defined like this:

// Structs have internal tags:
struct S1 {
    tag: TagS1,
    // ...
}
enum TagS1 {
    #[serde(rename = "s1")]
    S1
}

// Enums  have a single string variant and many internally tagged structs.
#[serde(untagged)]
enum E1 {
    String(String),
    S1(Box<S1>),
    // ...
}

I am parsing 13kb of JSON, and it results in stack overflow with 8MB of stack on macOS.

If I change it to 16MB using export RUST_MIN_STACK=16777216 && ... everything is fine.

Why does this require a stack of 8MB+? Is it just what is required, or is the way I'm using untagged causing issues?

Would this use less stack if I was using the TryFrom<Value> and Into<Value> Serde attributes on the enum to map my tags directly to an exact type (instead of untagged searching for the correct variant)?

I have flame graph's of both cases if thats useful.

Related:
#1062 (comment)

• More than 8MB is reasonable for types of size 1MB.

[Mingun]

I think this is happened because of internal buffering (using Content type) of everything on each level of structs. Even if content is already buffered it will be buffered once again on the next nested level.

[dtolnay]

#2148 made some improvements for recursive untagged types which should be applicable here. I'll close the issue since I don't plan to reproduce/investigate this, but if someone looks into this further to make an assessment of what part of this is leading to excessive stack usage, they should feel free to open a new issue with a minimal compilable example to reproduce the behavior.