You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Indexes datastructure are fully copied into memory when parsing an index.
There is no limit on how big thoses copies were (because practical indexes used by peoples are bigger than what would be a safe limit).
So if you parse an index from untrusted user input, they can send you a really big index and memory hog or OOM you.
Would be nice if we have a way to parse untrusted indexes without opening yourself to OOMs.
go-car@v2.4.0 also includes additional documentation regarding the dangers of consuming CARv2 index data from untrusted sources and a recommendation to regenerate indexes of CAR data from such sources where an index is required.
The text was updated successfully, but these errors were encountered:
The index.ReadFrom tests are moved to run explicitly against that function instead of running as part of inspection. Once the streaming index read is implemented and used as part of inspection, those tests should also run as part of inspection as it was before.
@rvagg we do not use indexes in Kubo, and I have already too much things to do.
I cannot take on that work sorry.
I opened it because as far as I understood in the discussion we had indexes that allocates might be an issue, however boost was not able to do a refactor use io.ReaderAt instead (or / and was concerned of copying into bytes.Buffer due to performance issues I belive).
If consumers of indexes (boost) don't think this is an issue or don't care about it, then we can close this.
Indexes datastructure are fully copied into memory when parsing an index.
There is no limit on how big thoses copies were (because practical indexes used by peoples are bigger than what would be a safe limit).
So if you parse an index from untrusted user input, they can send you a really big index and memory hog or OOM you.
Would be nice if we have a way to parse untrusted indexes without opening yourself to OOMs.
This was first attempted in #312 (see discussion in https://github.com/ipld/go-car-priv/pull/2):
But was reverted due to API breaking considerations (see discussion in the channel and https://github.com/ipld/go-car-priv/pull/18)):
See GHSA-9x4h-8wgm-8xfg
The text was updated successfully, but these errors were encountered: