Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame.OrderBy methods incorrect behavior with null values #7102

Open
asmirnov82 opened this issue Mar 22, 2024 · 0 comments · May be fixed by #7118
Open

DataFrame.OrderBy methods incorrect behavior with null values #7102

asmirnov82 opened this issue Mar 22, 2024 · 0 comments · May be fixed by #7118
Labels
untriaged New issue has not been triaged

Comments

@asmirnov82
Copy link
Contributor

asmirnov82 commented Mar 22, 2024

DataFrame OrderBy method should always place null values at the bottom of the list (after not nullable values) independently of sorting (ascending or descending). This is how Python does and how DataFrameColumn.Sort method works.

To Reproduce:

var col1 = new Int32DataFrameColumn("Index", new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 });
var col2 = new StringDataFrameColumn("Country", new[] { "USA", "France", "UK", "Brazil", "Russia", "India", null, "China", null });
var col3 = new StringDataFrameColumn("Capital", new[] { "Washington", "Paris", "London", "Brasilia", "Moscow", "New Dehli", null, "Beijing", null});

var df = new DataFrame(col1, col2, col3);
Console.WriteLine(df.OrderByDescending("Capital"));

Actual behaiour:

Index Country Capital
9 null null
7 null null
1 USA Washington
2 France Paris
6 India New Dehli
5 Russia Moscow
3 UK London
4 Brazil Brasilia
8 China Beijing

Expected behaiour:

Index Country Capital
1 USA Washington
2 France Paris
6 India New Dehli
5 Russia Moscow
3 UK London
4 Brazil Brasilia
8 China Beijing
9 null null
7 null null

Notes:

'Console.WriteLine(new DataFrame([col3.Sort(ascending: false)]));' works correctly

Capital
Washington
Paris
New Dehli
Moscow
London
Brasilia
Beijing
null
null

Issue was already mention in https://github.com/dotnet/machinelearning/pull/5776/files#r624316355

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged New issue has not been triaged
Projects
None yet
1 participant