Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

min compute kernel is incorrect with sliced buffers in arrow 23 #2779

Closed
alamb opened this issue Sep 26, 2022 · 1 comment · Fixed by #2780
Closed

min compute kernel is incorrect with sliced buffers in arrow 23 #2779

alamb opened this issue Sep 26, 2022 · 1 comment · Fixed by #2780
Labels
arrow Changes to the arrow crate bug

Comments

@alamb
Copy link
Contributor

alamb commented Sep 26, 2022

Describe the bug

When operating on an array that was sliceed, the result of min is incorrect

min([Null, 4.0] --> should be 4.0

However,

min([Null, Null, Null, Null, Null, Null, 4.0].slice(4,2)) --> should be 4.0 as well but returns None

To Reproduce

Use this program:

use arrow::{array::{Float64Array, Array, as_primitive_array}, datatypes::Float64Type};



fn main() {
    // behavior of sliced min changed (TODO make a demonstration)

    // Input is [None, 4.0], min value should be 4.0
    let expected_min = Some(4.0);
    let input: Float64Array = vec![None, Some(4.0)].into_iter().collect();
    let min = arrow::compute::min(&input);

    assert_eq!(min, expected_min);

    // however, when sliced the min value is incorrect
    let sliced_input: Float64Array = vec![None, None, None, None, None, Some(4.0)].into_iter().collect();
    let sliced_input = sliced_input.slice(4, 2);
    let sliced_input = as_primitive_array::<Float64Type>(&sliced_input);

    // The array valare are equal
    assert_eq!(sliced_input, &input);

    // but the minimum is not
    let sliced_min = arrow::compute::min(&sliced_input);
    assert_eq!(sliced_min, expected_min);

}

Expected behavior
The example should pass. It passes on arrow 22.0.0 but fails on arrow 23.0.0 like this:

thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `None`,
 right: `Some(4.0)`', src/main.rs:25:5
stack backtrace:

Additional context
Found while debugging upgrade in IOx: https://github.com/influxdata/influxdb_iox/pull/5694

@alamb alamb added arrow Changes to the arrow crate bug labels Sep 26, 2022
@tustvold
Copy link
Contributor

Fix incoming

tustvold added a commit to tustvold/arrow-rs that referenced this issue Sep 26, 2022
tustvold added a commit that referenced this issue Sep 26, 2022
* Don't apply array offset twice (#2779)

* More tests
alamb pushed a commit to alamb/arrow-rs that referenced this issue Sep 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants