Skip to content

feat: use binary COPY for Postgres insert#34

Open
lewiszlw wants to merge 1 commit into
masterfrom
task-c505-pg-insert
Open

feat: use binary COPY for Postgres insert#34
lewiszlw wants to merge 1 commit into
masterfrom
task-c505-pg-insert

Conversation

@lewiszlw

Copy link
Copy Markdown
Member

Summary

Replace the text-based INSERT SQL approach with Postgres binary COPY protocol via BinaryCopyInWriter. This improves performance and correctness for complex data types.

Key Changes

  • Rewrite insert() to use COPY ... FROM STDIN BINARY
  • Add PgValue enum implementing ToSql for all supported types (Bool/Int/Float/Varchar/Bytea/Uuid/Json/Jsonb/Timestamp/TimestampTz/Date/Time/Interval/Numeric/Array)
  • Add �rray_value_to_pg_value() helper to convert Arrow Array elements to PgValue
  • Add custom write_numeric_to_sql() for NUMERIC binary encoding (since �igdecimal does not have built-in ToSql support)
  • Add ormat_decimal_i128() / ormat_decimal_i256() helpers for correct Arrow Decimal conversion
  • Fix JSONB binary format (add version prefix byte

Replace the text-based INSERT SQL approach with Postgres binary COPY
protocol via BinaryCopyInWriter. This improves performance and
correctness for complex data types.

Key changes:
- Rewrite insert() to use COPY ... FROM STDIN BINARY
- Add PgValue enum implementing ToSql for all supported types
- Add custom write_numeric_to_sql for NUMERIC binary encoding
- Fix JSONB binary format (version prefix byte)
- Fix interval binary encoding byte order
- Add format_decimal_i128/i256 helpers for correct Decimal conversion

All 19 postgres integration tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant