Billion laughs attack

[kislyuk]

An anonymous security researcher has been threatening to advertise a fact that they pointed out to me, so I'm going to post a summary here. yq is susceptible to "billion laughs" resource exhaustion attacks, due to the fact that the underlying YAML parsing library, PyYAML, is susceptible to it when handling YAML anchors. While PyYAML disclaims responsibility for implementing mitigations for such an attack, it's possible to design an interface that would prevent this problem. Even though the data structure that PyYAML loads into memory when a malicious document is parsed is as compact as the document itself, any user of PyYAML trying to serialize or exhaustively traverse that document will trigger the attack. Because such a serialization is necessary when translating YAML to JSON, yq is susceptible.

Users of yq should be aware that yq is not hardened against malicious YAML documents and does not incorporate mitigations for this type of attack. While no further vectors are known, a DoS attack may be possible if untrusted YAML input is passed to yq by an online service.

Two types of mitigations have been implemented for this type of issue: refusing to dereference anchors (as seen in GitHub Actions and lxml) and flattening (deep-copying) the full document while monitoring memory usage, and raising an error if it exceeds some multiple of what is expected based on the serialized document size (as seen in go-yaml).

Because of PyYAML's refusal to address this and other issues, it is likely that yq will have to switch to a different YAML parser before this can be addressed. One solution would be to switch to a more principled parser like StrictYAML, but that would require porting the existing parser customizations to the StrictYAML API. This would eliminate anchor support altogether instead of making it optional or detecting malicious levels of expansion. For the latter, I'm not aware of any Python-based tools that would allow this.

[kislyuk]

Protections against YAML entity expansion attacks were added in v2.13.0.

[0x2b3bfa0]

Related commits from v2.12.2...v2.13.0

• 1929c63 Disable anchors
• a5d2c57 Use CSafeLoader where available, begin configurable anchor handling
• 7c549ea Add test for entity expansion
• 57773ab Wire up expand_aliases
• 08bef55 Partial roundtrip of anchors and aliases