Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EnabledStatistics::None not working #2185

Closed
alanhe opened this issue Jul 27, 2022 · 1 comment · Fixed by #2191
Closed

EnabledStatistics::None not working #2185

alanhe opened this issue Jul 27, 2022 · 1 comment · Fixed by #2191
Labels
bug parquet Changes to the parquet crate

Comments

@alanhe
Copy link

alanhe commented Jul 27, 2022

Describe the bug
Take the code as an example,

fn main() {
    let root_schema = Arc::new(Schema::new(vec![Field::new("a", DataType::Int64, false)]));

    let props = WriterProperties::builder()
        .set_statistics_enabled(EnabledStatistics::None)
        .build();

    let file = File::create("test.parquet").unwrap();
    let mut writer = ArrowWriter::try_new(
        file.try_clone().unwrap(),
        Arc::clone(&root_schema),
        Some(props),
    )
    .unwrap();

    let a = Int64Array::from(vec![1, 2, 3, 4, 5]);
    let batch = RecordBatch::try_new(Arc::clone(&root_schema), vec![Arc::new(a)]).unwrap();

    writer.write(&batch);
    writer.close();
}

When the default statistics_enabled is set to EnabledStatistics::None, if I read the metadata of test.parquet with code,

let f = File::open("test.parquet").unwrap();
let r = ParquetFileArrowReader::try_new(f).unwrap();
let m = r.metadata();
println!("{:#?}", m);

I get:

...
statistics: Some(
  Int64(
    {min: Some(1), max: Some(5), distinct_count: None, null_count: 0, min_max_deprecated: false},
  ),
 ),
...

Expected behavior

...
statistics: None
...

Additional context
Tested against arrow 19.0.0.

@alanhe alanhe added the bug label Jul 27, 2022
@tustvold
Copy link
Contributor

Oops, can confirm this was introduced by #2045

tustvold added a commit to tustvold/arrow-rs that referenced this issue Jul 27, 2022
@alamb alamb added the parquet Changes to the parquet crate label Aug 5, 2022
@alamb alamb changed the title EnabledStatistics::None not working EnabledStatistics::None not working Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants