Receipt organization and preview performance

Measured on 2026-10-06 in an isolated Python 3.12 environment on the Linux host. These are SQLite/Pillow measurements, not Android UI frame times or ML Kit results.

Changes

  • SQLite search covers display name/merchant, group, ISO date, and two-decimal amount. Text is literal (including %, _, and quotes); SQL values are parameterized. Default results are bounded at 200, with deterministic newest/oldest/amount/name sorting. Optional inclusive date, group, and amount filters are validated in ReceiptService.
  • Home summaries aggregate the current calendar month and fetch only five recent receipts. Group totals include empty groups. Renaming a custom group retains its ID and receipt links. General remains the stable default. Existing receipt editing continues to move groups.
  • Repository connections now close deterministically after commit or rollback. A sqlite3 connection’s built-in context manager alone does not close the connection.
  • Gallery preview preparation produces a maximum-512-pixel JPEG on background workers. Cache keys include source path, modification timestamp and file size. Replacements invalidate old previews; deleting an image removes its preview. Atomic writes keep concurrent readers away from partial files. Generation/deletion are serialized by the shared storage service; different preview sizes have separate keys. A source changed during decoding is not published under the old cache version. Originals remain unchanged for OCR/export/zoom.
  • Android source packaging excludes local agent/configuration directories and reports.

No schema change was needed. Existing date and group indexes already serve short queries. The group/date query used a temporary sort but took only 0.42 ms at baseline; adding an index was not justified by that measurement. Older databases still receive the existing additive display-name/group migration, now explicitly tested for repeatable upgrades.

Measurements

Before: data-baseline.json. After, including per-image measurements: data-after.json. Repeatable comparison: browsing-reproducible.json.

Operation Before After Scope
Initial browse database fetch 20.276 ms 0.490 ms 10,000 synthetic receipts; all rows before, first 200 after
Merchant search unavailable 3.286 ms 10,000 synthetic rows, bounded matches
Monthly summary + recent five unavailable 0.618 ms 10,000 synthetic rows
Preview decode, mean of image medians 9.102 ms 0.477 ms Ten real fixture images; after includes warm cache lookup
Cold thumbnail preparation none 16.640 ms Mean over ten images; additional first-use worker cost

Query measurements use 15 repetitions; previews use eight per image. Query p95 is the nearest-rank p95 (the largest sample for 15 repetitions). These small samples support local comparisons, not production latency guarantees. Warm preview work is ~95% lower; there is an explicit cold-generation cost, and UI/device results require separate verification.

Reproduce without accessing personal data:

python tools/benchmark_browsing.py --output /tmp/papernest-browsing.json
python -m unittest discover -s tests -p 'test_receipt_organization.py'
python -m unittest discover -s tests -p 'test_receipt_thumbnails.py'

Validation

11 organization tests pass: merchant/group/date/amount search, literal query safety, sorting, limits, combined filters, invalid ranges, monthly boundaries, group rename, duplicate-name rollback, default preservation, moving/deleting receipts, connection closure/rollback, and repeatable legacy migration retaining original IDs/images/amounts.

Eight thumbnail tests pass: original checksum/content preservation, reduced pixel dimensions, warm lookup without decoding, replacement invalidation, EXIF orientation/content, missing or corrupt image recovery, temporary-file cleanup, cache deletion with the source, concurrent preview sizes, and replacement during decoding.

Remaining scope: SQLite’s built-in case-insensitive matching is ASCII-based. Preview cache contains one version per source/size; it is not a global disk-quota cache. Android UI responsiveness and phone timing are tracked in the separate device verification report.