You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that pivot.frames().toDataFrame()
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process
The text was updated successfully, but these errors were encountered:
While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that
pivot.frames().toDataFrame()
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process
The text was updated successfully, but these errors were encountered: