CHANGELOG.md

Changelog

21.0.0 (2022-08-18)

Full Changelog

Breaking changes:

• Return structured ColumnCloseResult (#2465) #2466 [parquet] (tustvold)
• Push ChunkReader into SerializedPageReader (#2463) #2464 [parquet] (tustvold)
• Revise FromIterator for Decimal128Array to use Into instead of Borrow #2442 [parquet] [arrow] (viirya)
• Use Fixed-Length Array in BasicDecimal new and raw_value #2405 [arrow] (HaoYang670)
• Remove deprecated ParquetWriter #2380 [parquet] (tustvold)
• Remove deprecated SliceableCursor and InMemoryWriteableCursor #2378 [parquet] (tustvold)

Implemented enhancements:

• add into_inner method to ArrowWriter #2491 [parquet]
• Remove byteorder dependency #2472 [parquet]
• Return Structured ColumnCloseResult from GenericColumnWriter::close #2465 [parquet]
• Push ChunkReader into SerializedPageReader #2463 [parquet]
• Support SerializedPageReader::skip_page without OffsetIndex #2459 [parquet]
• Support Time64/Time32 comparison #2457 [arrow]
• Revise FromIterator for Decimal128Array to use Into instead of Borrow #2441 [parquet]
• Support RowFilter withinParquetRecordBatchReader #2431 [parquet]
• Remove the field StructBuilder::len #2429 [arrow]
• Standardize creation and configuration of parquet --> Arrow readers ( ParquetRecordBatchReaderBuilder) #2427 [parquet]
• Use OffsetIndex to Prune IO in ParquetRecordBatchStream #2426 [parquet]
• Support peek_next_page and skip_next_page in InMemoryPageReader #2406 [parquet]
• Support casting from Utf8/LargeUtf8 to Binary/LargeBinary #2402 [arrow]
• Support casting between Decimal128 and Decimal256 arrays #2375 [arrow]
• Combine multiple selections into the same batch size in skip_records #2358 [parquet]
• Add API to change timezone for timestamp array #2346 [arrow]
• Change the output of read_buffer Arrow IPC API to return Result<_> #2342 [arrow]
• Allow skip_records in GenericColumnReader to skip across row groups #2331 [parquet]
• Optimize the validation of Decimal256 #2320 [arrow]
• Implement Skip for DeltaBitPackDecoder #2281 [parquet]
• Changes to ParquetRecordBatchStream to support row filtering in DataFusion #2270 [parquet]
• Add ArrayReader::skip_records API #2197 [parquet]

Fixed bugs:

• Panic in SerializedPageReader without offset index #2503 [parquet]
• MapArray columns don't handle null values correctly #2484 [arrow]
• There is no compiler error when using an invalid Decimal type. #2440 [arrow]
• Flight SQL Server sends incorrect response for DoPutUpdateResult #2403 [arrow-flight]
• AsyncFileReaderNo Longer Object-Safe #2372 [parquet]
• StructBuilder Does not Verify Child Lengths #2252 [arrow]

Closed issues:

• Combine DecimalArray validation #2447 [arrow]

Merged pull requests:

• Fix bug in page skipping #2504 [parquet] (thinkharderdev)
• Fix MapArrayReader (#2484) (#1699) (#1561) #2500 [parquet] (tustvold)
• Add API to Retrieve Finished Writer from Parquet Writer #2498 [parquet] (jiacai2050)
• Derive Copy,Clone for BasicDecimal #2495 [arrow] (tustvold)
• remove byteorder dependency from parquet #2486 [parquet] (psvri)
• parquet-read: add support to read parquet data from stdin #2482 [parquet] (nvartolomei)
• Remove Position trait (#1163) #2479 [parquet] (tustvold)
• Add ChunkReader::get_bytes #2478 [parquet] (tustvold)
• RFC: Simplify decimal (#2440) #2477 [arrow] (tustvold)
• Use Parquet OffsetIndex to prune IO with RowSelection #2473 [parquet] (thinkharderdev)
• Remove unnecessary Option from Int96 #2471 [parquet] (tustvold)
• remove len field from StructBuilder #2468 [arrow] (psvri)
• Make Parquet reader filter APIs public (#1792) #2467 [parquet] (tustvold)
• enable ipc compression feature for integration test #2462 (liukun4515)
• Simplify implementation of Schema #2461 [arrow] (HaoYang670)
• Support skip_page missing OffsetIndex Fallback in SerializedPageReader #2460 [parquet] (Ted-Jiang)
• support time32/time64 comparison #2458 [arrow] (waitingkuo)
• Utf8array casting #2456 [arrow] (psvri)
• Remove outdated license text #2455 (alamb)
• Support RowFilter within ParquetRecordBatchReader (#2431) #2452 [parquet] (tustvold)
• benchmark: decimal builder and vec to decimal array #2450 [arrow] (liukun4515)
• Collocate Decimal Array Validation Logic #2446 [arrow] (liukun4515)
• Minor: Move From trait for Decimal256 impl to decimal.rs #2443 [arrow] (liukun4515)
• decimal benchmark: arrow reader decimal from parquet int32 and int64 #2438 [parquet] (liukun4515)
• MINOR: Simplify split_second function #2436 [arrow] (viirya)
• Add ParquetRecordBatchReaderBuilder (#2427) #2435 [parquet] (tustvold)
• refactor: refine validation for decimal128 array #2428 [arrow] (liukun4515)
• Benchmark of casting decimal arrays #2424 [arrow] (viirya)
• Test non-annotated repeated fields (#2394) #2422 [parquet] (tustvold)
• Fix #2416 Automatic version updates for github actions with dependabot #2417 (iemejia)
• Add validation logic for StructBuilder::finish #2413 [arrow] (psvri)
• test: add test for reading decimal value from primitive array reader #2411 [parquet] (liukun4515)
• Upgrade ahash to 0.8 #2410 [parquet] [arrow] (Dandandan)
• Support peek_next_page and skip_next_page in InMemoryPageReader #2407 [parquet] (Ted-Jiang)
• Fix DoPutUpdateResult #2404 [arrow-flight] (avantgardnerio)
• Implement Skip for DeltaBitPackDecoder #2393 [parquet] (Ted-Jiang)
• fix: Don't instantiate the scalar composition code quadratically for dictionaries #2391 [arrow] (Marwes)
• MINOR: Remove unused trait and some cleanup #2389 [arrow] (viirya)
• Decouple parquet fuzz tests from converter (#1661) #2386 [parquet] (tustvold)
• Rewrite Decimal and DecimalArray using const_generic #2383 [parquet] [arrow] (HaoYang670)
• Simplify BitReader (~5-10% faster) #2381 [parquet] (tustvold)
• Fix parquet clippy lints (#1254) #2377 [parquet] (tustvold)
• Cast between Decimal128 and Decimal256 arrays #2376 [arrow] (viirya)
• support compression for IPC with revamped feature flags #2369 [arrow] (alamb)
• Implement AsyncFileReader for Box<dyn AsyncFileReader> #2368 [parquet] (tustvold)
• Remove get_byte_ranges where bound #2366 [parquet] (tustvold)
• refactor: Make read_num_bytes a function instead of a macro #2364 [parquet] (Marwes)
• refactor: Group metrics into page and column metrics structs #2363 [parquet] (Marwes)
• Speed up Decimal256 validation based on bytes comparison and add benchmark test #2360 [parquet] [arrow] (liukun4515)
• Combine multiple selections into the same batch size in skip_records #2359 [parquet] (Ted-Jiang)
• Add API to change timezone for timestamp array #2347 [arrow] (viirya)
• Clean the code in field.rs and add more tests #2345 [arrow] (HaoYang670)
• Add Parquet RowFilter API #2335 [parquet] (tustvold)
• Make skip_records in complex_object_array can skip cross row groups #2332 [parquet] (Ted-Jiang)
• Integrate Record Skipping into Column Reader Fuzz Test #2315 [parquet] (Ted-Jiang)

* This Changelog was automatically generated by github_changelog_generator