Add Substring_by_char

[HaoYang670]

Which issue does this PR close?

Closes #1768.

Rationale for this change

Support substring by char

What changes are included in this PR?

1. new pub function substring_by_char
2. tests
3. benchmark
4. update docs

Performance

substring utf8 (start = 0, length = None)                                                                            
                        time:   [20.035 ms 20.056 ms 20.081 ms]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

substring utf8 (start = 1, length = str_len - 1)                                                                            
                        time:   [21.334 ms 21.358 ms 21.388 ms]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

substring utf8 by char  time:   [117.18 ms 117.29 ms 117.41 ms]                                   
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

1/6 of the speed of substring(by byte)

Are there any user-facing changes?

Yes.

[codecov-commenter]

Codecov Report

Merging #1784 (fcc0ee3) into master (1a64677) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1784      +/-   ##
==========================================
+ Coverage   83.39%   83.42%   +0.02%     
==========================================
  Files         198      198              
  Lines       56142    56303     +161     
==========================================
+ Hits        46822    46969     +147     
- Misses       9320     9334      +14     

Impacted Files	Coverage Δ
arrow/src/compute/kernels/substring.rs	99.51% <100.00%> (+0.09%)	⬆️
parquet/src/util/cursor.rs	62.18% <0.00%> (-1.69%)	⬇️
parquet/src/file/serialized_reader.rs	94.46% <0.00%> (-1.17%)	⬇️
arrow/src/datatypes/datatype.rs	65.42% <0.00%> (-0.38%)	⬇️
parquet_derive/src/parquet_field.rs	65.98% <0.00%> (-0.23%)	⬇️
parquet/src/encodings/encoding.rs	93.46% <0.00%> (-0.20%)	⬇️
arrow/src/ipc/reader.rs	90.73% <0.00%> (-0.11%)	⬇️
parquet/src/file/writer.rs	92.84% <0.00%> (-0.02%)	⬇️
parquet/src/arrow/arrow_writer.rs	97.76% <0.00%> (-0.02%)	⬇️
parquet/src/arrow/schema.rs	96.81% <0.00%> (-0.01%)	⬇️
... and 13 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a64677...fcc0ee3. Read the comment docs.

[tustvold]

I think this looks good, left some comments on how it could be improved - but happy for this to be a follow up PR

[tustvold]

Allocating a string temporary, only to copy out of it, is likely a significant portion of the slow-down. That combined with the null handling.

This could definitely be handled as a separate PR, but you might want to consider doing something like (not tested).

let nulls = // align bitmap to 0 offset, copying if already aligned
let mut vals = BufferBuilder::<u8>::new(array.value_data().len());
let mut indexes = BufferBuilder::<OffsetSize>::new(array.len() + 1);
indexes.append(0);

for val in array.iter() {
  let char_count = val.chars().count();
  let start = if start >= 0 {
      start.to_usize().unwrap().min(char_count)
  } else {
      char_count - (-start).to_usize().unwrap().min(char_count)
  };
  let length = length.map_or(char_count - start, |length| {
      length.to_usize().unwrap().min(char_count - start)
  });

  let mut start_byte = 0;
  let mut end_byte = val.len();
  for ((idx, (byte_idx, _)) in val.char_indices().enumerate() {
    if idx == start {
      start_byte = byte_idx;
    } else if idx == start + length {
      end_byte = byte_idx;
      break
    }
  }
  // Could even be unchecked
  vals.append_slice(&val[start_byte..end_byte]);
  indexes.append(vals.len() as _);
}

let data = ArrayDataBuilder::new(array.data_type()).len(array.len()).add_buffer(vals.finish())
  .add_buffer(indexes.finish()).add_buffer(vals.finish());

Ok(GenericStringArray::<OffsetSize>::from(unsafe {data.build_unchecked()}))

[HaoYang670]

Tracked by #1800

[tustvold]

Perhaps we could extract this as a function instead of duplicating it?

[HaoYang670]

Tracked by #1801

[tustvold]

I think it would be easier to follow to just construct the regular array, and then call array.slice(1, 2)

[HaoYang670]

Tracked by #1801

[tustvold]

Loving the test coverage 👍