Optimize import

.argent/flows/restore-pipeline-speedup-benchmark.yaml

@@ -0,0 +1,154 @@
+steps:
+  - echo: Restart the disposable API 37 emulator app and restore a selected
+      benchmark archive.
+  - launch: com.rajarsheechatterjee.LNReader.debug
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: DEVELOPMENT SERVERS
+  - echo: Select the configured local Metro server and wait for the reader screen.
+  - tap:
+      x: 0.5
+      y: 0.744
+  - echo: Dismiss the development menu to reach the reader home screen.
+  - tap:
+      text: Continue
+  - echo: Close the developer overlay and complete the fresh-install welcome setup.
+  - tap:
+      text: Close
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Welcome
+  - echo: Keep the default theme and finish onboarding to open the reader.
+  - tap:
+      id: button-text
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: New update available
+  - echo: Defer the optional app update and continue with the current debug build.
+  - tap:
+      id: button-text
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Search library
+  - echo: Open More to reach the local backup controls.
+  - tap:
+      x: 0.9
+      y: 0.929
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Task Queue
+  - echo: Open Settings and then the Backup screen.
+  - tap:
+      text: Settings
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Settings
+  - echo: Open the Backup settings screen.
+  - tap:
+      text: Backup
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Local Backup
+  - echo: Choose the local restore action and select the v2 benchmark archive in
+      Android's document picker.
+  - tap:
+      text: Restore backup
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Recent files
+  - echo: Open storage locations and navigate to Downloads.
+  - tap:
+      text: Show roots
+  - tap:
+      x: 0.335
+      y: 0.211
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: lnreader_backup.zip
+  - echo: Select the validated v2 test archive.
+  - tap:
+      id: android:id/title
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Changes already restored
+  - echo: Confirm the restore and wait for the local restore task to complete.
+  - tap:
+      x: 0.795
+      y: 0.578
+  - echo: Decline notifications because the restore benchmark does not need them.
+  - tap:
+      id: com.android.permissioncontroller:id/permission_deny_button
+  - echo: Open Task Queue and verify the restore result and progress.
+  - tap:
+      role: StaticText
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Settings
+  - echo: Return to More and open the queue.
+  - tap:
+      role: StaticText
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Task Queue
+  - tap:
+      text: Task Queue
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: Local Restore
+
+  - tool: await-ui-element
+    args:
+      condition: visible
+      selector:
+        text: No running tasks
+  - tool: button
+    args:
+      button: back
+  - tap:
+      x: 0.1
+      y: 0.9293
+  - tap:
+      text: Search library
+  - tool: keyboard
+    args:
+      text: Shadow Slave
+  - tool: button
+    args:
+      button: back
+  - tap:
+      text: Currently Reading
+  - tool: gesture-swipe
+    args:
+      fromX: 0.9
+      fromY: 0.167
+      toX: 0.15
+      toY: 0.167
+      durationMs: 300
+  - tap:
+      text: Done Downloading

.argent/flows/restore-v3-benchmark.yaml

@@ -0,0 +1,95 @@
+executionPrerequisite: LNReader's Android development build is on the DEVELOPMENT SERVERS screen, Metro is running at http://192.168.0.214:8081, and the matched v3 archive is in Downloads as lnreader_backup_v3.zip.
+
+steps:
+  - echo: Connect the fresh development client to the local Metro server.
+  - tap:
+      text: http://
+  - tool: keyboard
+    args:
+      text: http://192.168.0.214:8081
+  - tap:
+      text: Connect
+  - await: { visible: { text: Continue } }
+  - await: { idle: true }
+  - tap:
+      text: Continue
+  - await: { visible: { text: Close } }
+  - await: { idle: true }
+  - tap:
+      text: Close
+  - await: { visible: { text: Welcome } }
+  - await: { idle: true }
+  - echo: Keep the default theme and complete onboarding.
+  - tap:
+      text: Complete
+  - await: { visible: { text: New update available } }
+  - await: { idle: true }
+  - echo: Defer the optional update and use the current debug build.
+  - tap:
+      text: Later
+  - await: { visible: { text: Search library } }
+  - await: { idle: true }
+  - echo: Open More, then Settings and Backup.
+  - tap:
+      text: More
+  - await: { visible: { text: Settings } }
+  - await: { idle: true }
+  - tap:
+      text: Settings
+  - await: { visible: { id: appbar-content-title-text, text: Settings } }
+  - await: { idle: true }
+  - echo: Open the Backup settings screen.
+  - tap:
+      text: Backup
+  - await: { visible: { text: Local Backup } }
+  - await: { idle: true }
+  - tap:
+      text: Restore backup
+  - await: { visible: { text: lnreader_backup_v3.zip } }
+  - await: { idle: true }
+  - echo: Select the matched v3 archive; this fixture has downloaded-file restore disabled.
+  - tap:
+      id: android:id/title
+  - await: { visible: { text: Changes already restored } }
+  - await: { idle: true }
+  - echo: Confirm the restore and inspect its queued progress.
+  - tap:
+      text: Restore
+  - await:
+      visible:
+        id: 'com.android.permissioncontroller:id/permission_deny_button'
+      timeout: 15000
+  - await: { idle: true }
+  - echo: Decline notifications because this data restore does not use them.
+  - tap:
+      id: com.android.permissioncontroller:id/permission_deny_button
+  - await: { visible: { text: Local Backup } }
+  - await: { idle: true }
+  - echo: Return through Settings to the task queue while the restore runs.
+  - tool: button
+    args:
+      button: back
+  - await: { visible: { id: appbar-content-title-text, text: Settings } }
+  - await: { idle: true }
+  - echo: Return to More to open Task Queue.
+  - tool: button
+    args:
+      button: back
+  - await: { visible: { text: Task Queue } }
+  - await: { idle: true }
+  - echo: Open Task Queue and capture the running restore state.
+  - tap:
+      text: Task Queue
+  - await: { visible: { text: Local Restore } }
+  - await: { visible: { text: No running tasks }, timeout: 300000 }
+  - await: { idle: true }
+  - echo: Return to Library after the local restore task finishes.
+  - tool: button
+    args:
+      button: back
+  - await: { visible: { text: Task Queue } }
+  - await: { idle: true }
+  - tap:
+      text: Library
+  - await: { visible: { text: Search library } }
+  - await: { idle: true }

benchmark-comparison.md

@@ -0,0 +1,31 @@
+## Backup restore benchmark: v2 vs v3
+
+Compared the legacy v2 backup (`1.bench.zip`) with the new v3 backup (`2.bench.zip`). Both runs restored **7,323 novels** with **0 failures**, 9 categories, and 14 plugins.
+
+### End-to-end result
+
+| Metric | Legacy v2 | New v3 | Change |
+|---|---:|---:|---:|
+| Total restore time | 5m 15.6s | 4m 11.0s | **64.7s faster (20.5%)** |
+
+Total time is measured from `local:start` through `local:finalize:done`.
+
+### Restore phases
+
+| Phase | Legacy v2 | New v3 | Change |
+|---|---:|---:|---:|
+| Copy | 12.8s | 13.2s | 0.5s slower |
+| Outer unzip | 46.3s | 30.2s | **16.1s faster (34.7%)** |
+| Novel validation | 46.7s | 24.9s | **21.9s faster (46.8%)** |
+| Novel restore loop | 206.8s | 179.0s | **27.8s faster (13.4%)** |
+
+### Novel restore timing breakdown
+
+| Operation | Legacy v2 | New v3 | Change |
+|---|---:|---:|---:|
+| Read | 52.5s | 16.6s | **35.9s faster (68.4%)** |
+| Parse | 45.5s | 32.7s | **12.8s faster (28.2%)** |
+| Database | 135.5s | 136.7s | 1.3s slower (0.9%) |
+| Covers | 19.6s | 17.8s | 1.8s faster (9.0%) |
+
+The v3 run reduced end-to-end restore time by **about one minute**. The largest per-operation improvement was reading novel data, while database time was effectively unchanged. The phase durations and per-operation timings are reported separately from the benchmark logs; the operation timings should not be summed as if they were sequential phases.

benchmarks/README.md

@@ -0,0 +1,48 @@
+# Backup and restore benchmarks
+
+`backup-restore-full.txt` is the running record for full-library restore runs. `backup-restore-small.txt` holds reduced-archive runs. When the user supplies new benchmark output, add its results to the appropriate file in the same task. Treat the user’s log as the source of truth. Preserve the run label, backup format, code-path description, archive metadata, and any validation or image-optimization variant they identify. Do not invent missing details.
+
+## Entry format
+
+Append a dated section to the relevant file. Name the run by code path, backup format, and variant, for example `integrated restore-perf + Optimize Import v3, optimized images`.
+
+In `backup-restore-full.txt`, keep one consolidated table before the per-run detail entries. Add one row for each run discussed, including new user-provided runs. Label backup format, image variant, and code path separately; use `—` for missing values. Retain older entries rather than replacing them.
+
+For each per-run entry, include:
+
+- Input: backup format, novel count, chapter count, and selected sections when known.
+- Archive: entry count, compressed bytes, and uncompressed bytes when logged.
+- Total restore time, plus novel-pipeline and `restoreData` times when the events are present.
+- Copy, outer unzip, novel JSON read, parse, database, cover, category, settings, plugin, selected-file, and finalization timings when available.
+- Restored and failed novel/category counts, plugin count, and cover candidate/found/copied/missing/failure counts when available.
+- Database counters and validation/image variants when present.
+- A short comparison only where the runs are meaningfully matched. State which archive metadata matches. Label comparisons across different backup contents or code paths as run-level observations, not causal results.
+- When deriving a stage span from event timestamps rather than a logged `duration:` interval, label it as calculated and keep the raw event intervals alongside it.
+
+Keep raw measurements in milliseconds or seconds as emitted, with readable equivalents where helpful. Use decimal seconds to two places in summaries. Keep byte counts as bytes; optional MB figures must say whether they are decimal MB or MiB. Do not round away differences that matter to a comparison.
+
+## Log labels and shorthand
+
+- `v2` / `v3`: backup manifest format versions, not app or code versions.
+- `local:start` to `local:finalize:done`: end-to-end local restore duration reported by `Total time`.
+- `local:copy:done`: source archive copy into app cache has completed.
+- `local:outer-unzip:done`: the outer backup archive has been extracted. The JSON payload gives entry counts and compressed/uncompressed sizes when that instrumentation is available.
+- `restoreData`: manifest, library, categories, settings, and plugin-registry restore work.
+- `restoreData:novels:pipeline:done`: novel-file processing pipeline completion. `total` counts processed novel JSON files, not restored novel rows.
+- `readMs` / `parseMs` / `databaseMs` / `coverProcessingMs`: accumulated time for novel-file reads, JSON parsing, database restore batches, and cover processing. Cover time can include concurrent native copies; it is elapsed stage time, not a sum of per-file CPU time.
+- `databaseBatches` and `batchCount`: number of non-empty database restore batches in the emitting implementation.
+- `uniqueInputChapterCount`: Optimize Import deduplicates chapter identities before counting. Earlier restore-perf logs accumulated raw chapter rows despite using this same field name. Preserve the original key and explain its implementation-specific meaning.
+- `chapterWriteChunkCalls` / `chapterWriteRowsAttempted`: database insert statements and input rows submitted to those statements. Different implementations may batch these differently, so compare elapsed database time and workload before comparing counts.
+- `restoreFailures`: a bounded sample of per-novel database failures, not necessarily a complete count of all errors.
+- `restoreData:novels:validation:start`: a timing event only. Its `duration:` value measures the gap since the previous log event, not validation runtime; there is no matching validation-complete event in these logs.
+- `local:selected-archive-unzip:done`: extraction of one selected inner archive. `local:selected-archives:done` marks completion of those extractions.
+- `local:downloaded-files:done`: downloaded novel files have been restored. `local:selected-files:done` marks completion of selected-file handling.
+- Durations are wall-clock elapsed times from `performance.now()`. The `duration:` value is the interval since the preceding benchmark event. Adjacent stage intervals and end-to-end totals can differ slightly due to logging and omitted setup or completion work.
+
+## Comparisons and data quality
+
+Prefer runs with matching reported archive metadata, device, reset state, and workload. Matching file counts and byte counts mean the reported metadata matches; it does not prove the archives are byte-for-byte identical. Backup format, image optimization, validation settings, app code path, and run order can all affect timing. Record those distinctions explicitly.
+
+When showing percentage deltas, name the exact baseline in the table caption and calculate `(run value / baseline value - 1) × 100`. Positive values mean slower than baseline; negative values mean faster. If multiple Restore-perf runs exist, identify the specific one rather than calling it simply “Restore-perf”.
+
+Preserve each run separately. Do not replace older values with a new result or combine runs into an average unless the user asks for that analysis. If a log omits an archive size, code revision, device, or setting, write `not provided` rather than inferring it. When the user supplies more benchmark information, append it here and update this guide only if the format or shorthand changes.