Flip interval field ordering (#5654) #5656
Conversation
| @@ -1714,18 +1714,17 @@ mod tests { | |||
| IntervalDayTimeType::make_value(1, 3000), | |||
| // 90M milliseconds | |||
| IntervalDayTimeType::make_value(0, 90_000_000), | |||
| IntervalDayTimeType::make_value(4, 10), | |||
There was a problem hiding this comment.
The interval ordering is now even more arbitrary 😅
I suspect this is the most user-visible change, as I suspect not many people are manually bit-twiddling
|
Looks like the cast kernels will also need updating, but I'd like to get feedback on if this is even the right approach to take first. |
💯 |
|
You can probably add some PyArrow integration test(s) in https://github.com/apache/arrow-rs/blob/master/arrow-pyarrow-integration-testing/tests/test_sql.py ? Both to |
|
Thanks for opening this PR. Any idea on ETA for when this would be merged in? |
|
As this is a breaking change with potentially major downstream implications, I suspect it will likely be a while before we incorporate this PR, if even at all. The major potential pain point concerns IPC data files that have been written or are being sent over the network. We need to be very careful here. I suspect this will be a timeline in months I am afraid |
Which issue does this PR close?
Closes #5654
Rationale for this change
There have been reports that the definition we use is backwards, and whilst the specification is kind of ambiguous, we should probably follow what arrow-cpp does.
This is a draft as I think we should look to get integration test coverage in first to make sure that this is actually correct.
What changes are included in this PR?
Are there any user-facing changes?
Aside from the obvious there are a couple of potentially problematic changes that result from this:
The order was already arbitrary so the first may not be too problematic, but the latter is potentially. I do, however, suspect that few people are encoding intervals in IPC files else we would have had reports of this before