Skip to content

feat(manifest)!: Replace FetchEntries with iterator approach#985

Open
hdnpth wants to merge 3 commits intoapache:mainfrom
hdnpth:fix/manifest-iter-entries
Open

feat(manifest)!: Replace FetchEntries with iterator approach#985
hdnpth wants to merge 3 commits intoapache:mainfrom
hdnpth:fix/manifest-iter-entries

Conversation

@hdnpth
Copy link
Copy Markdown
Contributor

@hdnpth hdnpth commented May 5, 2026

FetchEntries materializes every manifest entry into a []ManifestEntry before returning, so a caller pays the full per-manifest entry slice even when it only needs to filter, count, or break out on first match. On tables with thousands of entries per manifest this shows up as avoidable memory pressure.

@hdnpth hdnpth requested a review from zeroshade as a code owner May 5, 2026 11:05
Comment thread manifest.go
Comment thread manifest.go Outdated
Comment thread manifest.go Outdated
Comment thread cmd/iceberg/output.go Outdated
Comment thread table/scanner.go Outdated
@hdnpth hdnpth changed the title feat(manifest): Replace FetchEntries with iterator approach feat(manifest)!: Replace FetchEntries with iterator approach May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants