Skip to content

[SPARK-57742][PYTHON] Add truncation disclaimer to unix_micros docstring#56853

Open
jubins wants to merge 1 commit into
apache:masterfrom
jubins:j-SPARK-57742-unix_micros-docstring
Open

[SPARK-57742][PYTHON] Add truncation disclaimer to unix_micros docstring#56853
jubins wants to merge 1 commit into
apache:masterfrom
jubins:j-SPARK-57742-unix_micros-docstring

Conversation

@jubins

@jubins jubins commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Fixes SPARK-57742 — adds "Truncates higher levels of precision." to the one-line description of unix_micros, making it consistent with unix_millis and unix_seconds which already carry this disclaimer.

This became a practical gap once unix_nanos (SPARK-57579) made nanosecond-precision timestamps (TIMESTAMP_LTZ(9) / TIMESTAMP_NTZ(9)) usable: a user passing such a column to unix_micros has no indication from the docstring that sub-microsecond digits will be silently dropped.

Brief change log

  • python/pyspark/sql/functions/builtin.py: appended "Truncates higher levels of precision." to the one-line summary of unix_micros, matching unix_millis and unix_seconds

Verifying this change

This is a documentation-only change. No behaviour is altered.

  • Confirm unix_micros docstring now reads: "Returns the number of microseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision."
  • Confirm unix_millis and unix_seconds already carry the same sentence (consistency check)

Does this pull request potentially affect one of the following parts

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public/@Evolving: no — docstring only
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: no
  • The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? No — this is a docstring fix.

Was generative AI tooling used to co-author this PR?

  • Yes — Claude Code was used as a pair-programming assistant. All changes were reviewed and verified by the author. Generated-by: Claude Opus 4.8

### What is the purpose of the change

Fixes SPARK-57742 — adds `"Truncates higher levels of precision."` to the one-line description of `unix_micros`, making it consistent with `unix_millis` and `unix_seconds` which already carry this disclaimer.

This became a practical gap once `unix_nanos` (SPARK-57579) made nanosecond-precision timestamps (`TIMESTAMP_LTZ(9)` / `TIMESTAMP_NTZ(9)`) usable: a user passing such a column to `unix_micros` has no indication from the docstring that sub-microsecond digits will be silently dropped.

### Brief change log

- `python/pyspark/sql/functions/builtin.py`: appended `"Truncates higher levels of precision."` to the one-line summary of `unix_micros`, matching `unix_millis` and `unix_seconds`

### Verifying this change

This is a documentation-only change. No behaviour is altered.

- [ ] Confirm `unix_micros` docstring now reads: `"Returns the number of microseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision."`
- [ ] Confirm `unix_millis` and `unix_seconds` already carry the same sentence (consistency check)

### Does this pull request potentially affect one of the following parts

- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public`/`@Evolving`: no — docstring only
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: no
- The S3 file system connector: no

### Documentation

Does this pull request introduce a new feature? No — this is a docstring fix.

### Was generative AI tooling used to co-author this PR?

Yes — Claude Code was used as a pair-programming assistant. All changes were reviewed and verified by the author.
Generated-by: Claude Opus 4.8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant