Repository navigation
fix(export): keep every value through an export and its import - #3707
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FormatPluginTests/ExportImportRoundTripTests: a SQLite table with every storage class, NULL, quotes, delimiters, LF/CRLF/CR line breaks, Unicode, a control character and binary goes out through each export plugin and back in through its import plugin. CSV (every delimiter and line ending, BOM, NULL spelled out), JSON and NDJSON, XLSX, SQL (every insert mode), SQL gzipped, SQL split into 1 MB parts restored in order.FormatLimitsnames what each format cannot carry.SQLiteLinkageonFormatPluginTests), read rows the way the SQLite driver does, and insert them the wayImportDataSinkAdapterdoes.scripts/probes/check-export-import-roundtrip.sh: the same round trip on MySQL/MariaDB or PostgreSQL with 200,000 rows (110 to 172 MB per file), through the built driver and format plugins. Row count and a server-side checksum on both sides. SQL dumps restore through the engine's own client.-5was written'-5, and PostgreSQL refused it on import. Same rule as the Linux app'sis_plain_number.\r\nis one Character, so a value with a Windows line break was written unquoted and split its row, which also put the text after the break, a formula included, at the start of a new row. Fixed for the CSV export, the MCP export, Copy as CSV, the structure grid copy and the activity log export, which now shareCsvRowConverter.encodeandPluginRowWriters.csvField.trueandfalseare written bare only in a boolean column. TextTRUEin a text column came backtrue. . A raw one read back as a line feed.PluginRowWritersTests:-1and+1now stay as they are, changed on purpose.Test plan
verify.sh test, 16 suites, 180 cases: pass. Without the fixes, 6 of the 10 round-trip tests fail (CSV x3, JSON x2, XLSX).verify.sh ios ClipboardExporterPasteboardTests: pass (PluginKit compiles in the iOS app).'-49998.999999and XLSX came back with a different checksum.Not in this PR
ClipboardExporterhas the samecontains("\n")check.NaNorInfinityis written as a number cell,<t>has noxml:space="preserve", control characters are dropped rather than written_xHHHH_. Excel's reading of each could not be checked here.''and NULL back as NULL; binary goes out as hex text (CSV, XLSX) or base64 (JSON). A decimal-comma CSV export has no import option to read it back.SQLFileParser, the app's SQL import reader, is app code neither the plugin test bundle nor the probe reaches.Fixes #3666