Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory allocation of 96543189696 bytes failed #124

Open
aminya opened this issue Sep 22, 2023 · 5 comments
Open

memory allocation of 96543189696 bytes failed #124

aminya opened this issue Sep 22, 2023 · 5 comments

Comments

@aminya
Copy link

aminya commented Sep 22, 2023

When an object is big, the package fails to allocate its debug representation of it in the memory, so it fails.

assert_eq!(big_obj, big_obj2)

The solution would be to act like the standard assert_eq, which doesn't fail or maybe represent the needed subset only.
Related to #19

@tommilligan
Copy link
Collaborator

It's questionable how we proceed here - as we do more processing that the standard assert_eq, there will always be some point where we run out of memory faster and panic.

There's probably some tweaking we can do, but my understanding is that we'll have to allocate a contiguous range of memory for these debug implementations somewhere, plus the overhead of diffing, formatting etc.

I think think I would have to know more about this 96.5GB representation, and why you want to use assert_eq to diff it's debug format, as that's well outside what this library is targeted at.

@aminya
Copy link
Author

aminya commented Sep 22, 2023

It is just a indextree with nested children. The object itself is not crazy big, but the Debug representation seems to be way outside the memory limits.

@aminya
Copy link
Author

aminya commented Sep 24, 2023

I have noticed that this crate shows the Debug representation of the two objects fully before showing the diff. I suspect that's why it fails. I only care about the diff and not the whole object.

Related to #114

@cschramm
Copy link

The reason for this is the use of diff::chars. It allocates a Vec of left.len() * right.len() i32s. The data compared here is probably just around 155 kB, leading to an insane 90 GiB Vec. The logic already starts getting a problem with just a couple of kB.

@aminya
Copy link
Author

aminya commented Jan 31, 2024

@cschramm Opened a ticket upstream to discuss this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants