Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pivot rendering in the Koltin notebook plugin is slow #674

Open
ermolenkodev opened this issue Apr 25, 2024 · 0 comments
Open

Pivot rendering in the Koltin notebook plugin is slow #674

ermolenkodev opened this issue Apr 25, 2024 · 0 comments
Assignees
Milestone

Comments

@ermolenkodev
Copy link
Contributor

While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that pivot.frames().toDataFrame()
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process

@ermolenkodev ermolenkodev added this to the 0.13.+ milestone Apr 25, 2024
@ermolenkodev ermolenkodev self-assigned this Apr 25, 2024
@zaleslaw zaleslaw modified the milestones: 0.13.+, 0.14.0 Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants