CHANGELOG.md

Changelog

22.0.0 (2022-09-02)

Full Changelog

Breaking changes:

• Use total_cmp for floating value ordering and remove nan_ordering feature flag #2614 [arrow] (viirya)
• Gate dyn comparison of dictionary arrays behind dyn_cmp_dict #2597 [arrow] (tustvold)
• Move JsonSerializable to json module (#2300) #2595 [arrow] (tustvold)
• Decimal precision scale datatype change #2532 [parquet] [arrow] (psvri)
• Refactor PrimitiveBuilder Constructors #2518 [parquet] [arrow] (psvri)
• Refactoring DecimalBuilder constructors #2517 [arrow] (psvri)
• Refactor FixedSizeBinaryBuilder Constructors #2516 [parquet] [arrow] (psvri)
• Refactor BooleanBuilder Constructors #2515 [arrow] (psvri)
• Refactor UnionBuilder Constructors #2488 [arrow] (psvri)

Implemented enhancements:

• Add Macros to assist with static dispatch #2635 [arrow]
• Support comparison between DictionaryArray and BooleanArray #2617 [arrow]
• Use total_cmp for floating value ordering and remove nan_ordering feature flag #2613 [arrow]
• Support empty projection in CSV, JSON readers #2603 [arrow]
• Support SQL-compliant NaN ordering between for DictionaryArray and non-DictionaryArray #2599 [arrow]
• Add dyn_cmp_dict feature flag to gate dyn comparison of dictionary arrays #2596 [arrow]
• Add max_dyn and min_dyn for max/min for dictionary array #2584 [arrow]
• Allow FlightSQL implementers to extend do_get() #2581 [arrow-flight]
• Support SQL-compliant behavior on eq_dyn, neq_dyn, lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #2569 [arrow]
• Add sql-compliant feature for enabling sql-compliant kernel behavior #2568
• Calculate sum for dictionary array #2565 [arrow]
• Add test for float nan comparison #2556 [arrow]
• Compare dictionary with string array #2548 [arrow]
• Compare dictionary with primitive array in lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #2538 [arrow]
• Compare dictionary with primitive array in eq_dyn and neq_dyn #2535 [arrow]
• UnionBuilder Create Children With Capacity #2523 [arrow]
• Speed up like_utf8_scalar for %pat% #2519 [arrow]
• Replace macro with TypedDictionaryArray in comparison kernels #2513 [arrow]
• Use same codebase for boolean kernels #2507 [arrow]
• Use u8 for Decimal Precision and Scale #2496 [arrow]
• Integrate skip row without pageIndex in SerializedPageReader in Fuzz Test #2475 [parquet]
• Avoid unecessary copies in Arrow IPC reader #2437 [arrow]
• Add GenericColumnReader::skip_records Missing OffsetIndex Fallback #2433 [parquet]
• Support Reading PageIndex with ParquetRecordBatchStream #2430 [parquet]
• Specialize FixedLenByteArrayReader for Parquet #2318 [parquet]
• Make JSON support Optional via Feature Flag #2300 [arrow]

Fixed bugs:

• Casting timestamp array to string should not ignore timezone #2607 [arrow]
• Ilike_ut8_scalar kernals have incorrect logic #2544 [arrow]
• Always validate the array data when creating array in IPC reader #2541 [arrow]
• Int96Converter Truncates Timestamps #2480 [parquet]
• Error Reading Page Index When Not Available #2434 [parquet]
• ParquetFileArrowReader::get_record_reader[_by_colum] batch_size overallocates #2321 [parquet]

Documentation updates:

• Document All Arrow Features in docs.rs #2633 [arrow]

Closed issues:

• Add support for CAST from Interval(DayTime) to Timestamp(Nanosecond, None) #2606 [arrow]
• Why do we check for null in TypedDictionaryArray value function #2564 [arrow]
• Add the length field for Buffer #2524 [arrow]
• Avoid large over allocate buffer in async reader #2512 [parquet]
• Rewriting Decimal Builders using const_generic. #2390 [arrow]
• Rewrite Decimal Array using const_generic #2384 [arrow]

Merged pull requests:

• Add downcast macros (#2635) #2636 [arrow] (tustvold)
• Document all arrow features in docs.rs (#2633) #2634 [arrow] (tustvold)
• Document dyn_cmp_dict #2624 [arrow] (tustvold)
• Support comparison between DictionaryArray and BooleanArray #2618 [arrow] (viirya)
• Cast timestamp array to string array with timezone #2608 [arrow] (viirya)
• Support empty projection in CSV and JSON readers #2604 [arrow] (Dandandan)
• Make JSON support optional via a feature flag (#2300) #2601 [parquet] [arrow] (tustvold)
• Support SQL-compliant NaN ordering for DictionaryArray and non-DictionaryArray #2600 [arrow] (viirya)
• Split out integration test plumbing (#2594) (#2300) #2598 [arrow] (tustvold)
• Refactor Binary Builder and String Builder Constructors #2592 [parquet] [arrow] (psvri)
• Dictionary like scalar kernels #2591 [arrow] (psvri)
• Validate dictionary key in TypedDictionaryArray (#2578) #2589 [arrow] (tustvold)
• Add max_dyn and min_dyn for max/min for dictionary array #2585 [arrow] (viirya)
• Code cleanup of array value functions #2583 [arrow] (psvri)
• Allow overriding of do_get & export useful macro #2582 [arrow-flight] (avantgardnerio)
• MINOR: Upgrade to pyo3 0.17 #2576 [arrow] (andygrove)
• Support SQL-compliant NaN behavior on eq_dyn, neq_dyn, lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #2570 [arrow] (viirya)
• Add sum_dyn to calculate sum for dictionary array #2566 [arrow] (viirya)
• struct UnionBuilder will create child buffers with capacity #2560 [arrow] (kastolars)
• Don't panic on RleValueEncoder::flush_buffer if empty (#2558) #2559 [parquet] (tustvold)
• Add the length field for Buffer and use more Buffer in IPC reader to avoid memory copy. #2557 [arrow] [arrow-flight] (HaoYang670)
• Add test for float nan comparison #2555 [arrow] (viirya)
• Compare dictionary array with string array #2549 [arrow] (viirya)
• Always validate the array data (except the Decimal) when creating array in IPC reader #2547 [arrow] (HaoYang670)
• MINOR: Fix test_row_type_validation test #2546 [arrow] (viirya)
• Fix ilike_utf8_scalar kernals #2545 [arrow] (psvri)
• fix typo #2540 (00Masato)
• Compare dictionary array and primitive array in lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn kernels #2539 [arrow] (viirya)
• [MINOR]Avoid large over allocate buffer in async reader #2537 [parquet] (Ted-Jiang)
• Compare dictionary with primitive array in eq_dyn and neq_dyn #2533 [arrow] (viirya)
• Add iterator for FixedSizeBinaryArray #2531 [arrow] (tustvold)
• add bench: decimal with byte array and fixed length byte array #2529 [parquet] (liukun4515)
• Add FixedLengthByteArrayReader Remove ComplexObjectArrayReader #2528 [parquet] (tustvold)
• Split out byte array decoders (#2318) #2527 [parquet] (tustvold)
• Use offset index in ParquetRecordBatchStream #2526 [parquet] (thinkharderdev)
• Clean the create_array in IPC reader. #2525 [arrow] (HaoYang670)
• Remove DecimalByteArrayConvert (#2480) #2522 [parquet] (tustvold)
• Improve performance of %pat% (>3x speedup) #2521 [arrow] (Dandandan)
• remove len field from MapBuilder #2520 [arrow] (psvri)
• Replace macro with TypedDictionaryArray in comparison kernels #2514 [arrow] (viirya)
• Avoid large over allocate buffer in sync reader #2511 [parquet] (Ted-Jiang)
• Avoid useless memory copies in IPC reader. #2510 [arrow] (HaoYang670)
• Refactor boolean kernels to use same codebase #2508 [arrow] (viirya)
• Remove Int96Converter (#2480) #2481 [parquet] (tustvold)

* This Changelog was automatically generated by github_changelog_generator