feat:Add function for row alignment with page mask (#1791)

* add range and rowRanges * add some tests for range * add filter logic * fix fmt * fix todo * fix test * Apply suggestions from code review Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com> * fix compute_row_ranges * fix annotation * change to use std:ops:RangeInclusive * Apply suggestions from code review Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com> * fix * Update parquet/src/file/page_index/range.rs Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com> Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com> Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
apache · Jun 6, 2022 · 9b220a5 · 9b220a5
1 parent c8d4323
commit 9b220a5
Show file tree

Hide file tree

Showing 4 changed files with 477 additions and 0 deletions.
diff --git a/parquet/src/file/metadata.rs b/parquet/src/file/metadata.rs
@@ -223,6 +223,7 @@ pub struct RowGroupMetaData {
     num_rows: i64,
     total_byte_size: i64,
     schema_descr: SchemaDescPtr,
+    // Todo add filter result -> row range
 }
 
 impl RowGroupMetaData {

diff --git a/parquet/src/file/page_index/mod.rs b/parquet/src/file/page_index/mod.rs
@@ -17,3 +17,4 @@
 
 pub mod index;
 pub mod index_reader;
+pub(crate) mod range;