Tracking: Postgres sink issues

[StrikeW]

• Support sink to uuid array (uuid[]) column Support more array types in remote sinks #14616

  I find that it might be difficult to support this via JDBC interface, since java.sql.Types doesn't have a type corresponding to UUID array, so we cannot get the uuid array type from the prepared statement. I am thinking about rewriting the Postgres sink with Rust tokio_postgres and leverage the ScalarAdapter (refactor(cdc): refactor parsing of non-builtin Postgres data types #16589 ) to convert our types to PG types.

• Check data type mismatch of downstream table Catch and return sink writer error ERROR: column \“some_column\” is of type some_pg_type but expression is of type character varying` to user #16687

  We can check the data types after prepared the SQL statement

[fuyufjh]

I am thinking about rewriting the Postgres sink with Rust tokio_postgres and leverage the ScalarAdapter

+1 for migrating from JDBC to native implementation. My intuitive is that JDBC adds another abstraction layer and makes it more difficult for us to make all data type conversion consistent among source/sink systems. Let alone the performance loss.

[xiangjinwu]

• Agree to implement the sink in rust and avoid going thru JDBC.

• I find that it might be difficult to support this via JDBC interface, since java.sql.Types doesn't have a type corresponding to UUID array, so we cannot get the uuid array type from the prepared statement.

  It is doable. The following snippet outputs _uuid, which means array of uuid in PostgreSQL (just like _float8 is double precision[]).

    PreparedStatement stp = conn.prepareStatement("select array['018f761c-a819-748a-863f-52204f67aa2a'::uuid] = ?;"
    System.out.println(stp.getParameterMetaData().getParameterTypeName(1));
  

  Just sharing it is possible. But for a better future I am also in favor of the rust implementation.