Skip to content

Commit 6d6347c

Browse files
author
“Zhibin
committed
docs: Add missing parameter details in add_files documentatio (#2132)
1 parent 0841f54 commit 6d6347c

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

mkdocs/docs/api.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1015,6 +1015,9 @@ Because `add_files` uses existing files without writing new parquet files that a
10151015
!!! warning "Maintenance Operations"
10161016
Because `add_files` commits the existing parquet files to the Iceberg Table as any other data file, destructive maintenance operations like expiring snapshots will remove them.
10171017

1018+
!!! warning "Check Duplicate Files"
1019+
The `check_duplicate_files` parameter is `True` by default and will check the new files against the existing Iceberg table data files to prevent duplicates. This check can be expensive for large tables with many files. It is recommended to use the default configuration. The check can be turned off by setting `check_duplicate_files=False`, but this may result in duplicate files being added to the table, which can lead to data consistency issues and potential table corruption if the same data file is added multiple times.
1020+
10181021
<!-- prettier-ignore-end -->
10191022

10201023
### Usage

0 commit comments

Comments
 (0)