Refactor integer type inference logic to fit smallest type

[jondo2010]

Which issue does this PR close?

Partially fixes #802

Rationale for this change

As explained in #802, the current infer_schema logic always returns DataType::Int64 for all integer types.

This PR adds additional inferring logic to find the smallest integer type that fits the data.

This PR also fixes an issue where the data in fact doesn't fit in a DataType::Int64 and instead requires a UInt64.

Are there any user-facing changes?

Any user code loading CSV data using infer_schema() and always expecting to receive Int64 fields would be affected.

No API changes.

[Jefffrey]

Will need to swap out usage of lexical_core

I'll take a closer look at the PR later, just noting this down first 👍

[Jefffrey]

There was a recent PR which highlighted that lexical_core has a longstanding issue where it can parse overflow incorrectly: #5398

For example, given this example program:

fn main() {
    let a = lexical_core::parse::<u8>(b"999");
    println!("{:?}", a);
    let a = "999".parse::<u8>();
    println!("{:?}", a);
}

It returns:

Ok(231)
Err(ParseIntError { kind: PosOverflow })

Using lexical_core = "0.8.5".

I don't think we can rely on lexical_core anymore, at least not for Integer parsing

[jondo2010]

Ah that's unfortunate.

[tustvold]

I worry that this will make schema inference very fragile, as it is typically only run on a subset of the data and therefore with this change may be overly restrictive on size or sign.

A few thoughts:

• We should prefer signed types by default, as this most of the time is sufficient
• The new behaviour should be opt-in
• I think this will substantially regress performance as it now must parse integers

Perhaps we could see how other systems handle this, e.g. DuckDb or Spark?

[tustvold]

Marking as draft as not waiting on review, please feel free to mark as ready for review when you would like me to take another look