Skip to content

fix: convert pandas Timestamp to Snowflake-compatible string in write_pandas (closes #991) - #4316

Open
botbikamordehai2-sketch wants to merge 1 commit into
snowflakedb:mainfrom
botbikamordehai2-sketch:fix/issue-991-1786786034
Open

fix: convert pandas Timestamp to Snowflake-compatible string in write_pandas (closes #991)#4316
botbikamordehai2-sketch wants to merge 1 commit into
snowflakedb:mainfrom
botbikamordehai2-sketch:fix/issue-991-1786786034

Conversation

@botbikamordehai2-sketch

Copy link
Copy Markdown

What

When writing a pandas DataFrame with a datetime64[ns] column to Snowflake using write_pandas, the generated SQL includes timestamps in a format that Snowflake interprets as an invalid date (e.g., '2013-02-10 00:00:00' may be misformatted or include microseconds/timezone that Snowflake cannot parse). The issue is in _extract_schema_and_data_from_pandas_df where Timestamp objects are converted to strings without explicit formatting, causing invalid date errors.

Fix

In the _extract_schema_and_data_from_pandas_df function, when handling pd.Timestamp values, convert them to a string in the ISO 8601 format '%Y-%m-%d %H:%M:%S.%f' (or using '%Y-%m-%d %H:%M:%S' for non-microsecond) which is a valid Snowflake timestamp literal. For timezone-aware timestamps, first normalize to UTC and then remove the timezone to align with Snowflake's default behavior of ignoring timezone offsets. This ensures that write_pandas generates safe SQL literals that Snowflake can parse.

Closes #991

@github-actions

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SNOW-886649: write_pandas inserts datetime64[ns] to Snowflake as an Invalid Date

1 participant