Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SonyIngest

tests license: MIT python 3.11+

Pull a Sony camera card into one folder where photos and video sit together in the order you actually shot them — with every copy checksum-verified and nothing ever imported twice.

macOS. Command line, plus a double-clickable app.


The problem

Sony Alpha bodies split a single shoot across two unrelated trees on the card:

/Volumes/UNTITLED/
├── DCIM/100MSDCF/          stills:  .ARW  .JPG  .HIF
├── MP_ROOT/                legacy MP4 on older bodies, often empty
└── PRIVATE/M4ROOT/
    ├── CLIP/               video:   .MP4 (XAVC S/HS), .MXF on some bodies
    ├── THMBNL/             video thumbnails
    └── *.XML               per-clip sidecars: true capture time, codec, GPS

This is mandated by the DCF standard for stills plus Sony's own XAVC/AVCHD layout for video, and no camera menu setting merges them. Every downstream annoyance — browsing in Finder, importing to Photos or Lightroom — starts here.

Worse, dragging the folders together by hand doesn't fix the ordering, because stills and clips record their timestamps in different time frames (see below). Sort the merged folder by date and your clips land hours — sometimes a day — away from the photos shot beside them.

What you get

~/Pictures/Camera/
├── 20250703-200349_DSC00001.JPG
├── 20250703-200418_C0001.MP4      ← shot 29 seconds after the photo above
├── 20250703-200502_DSC00002.JPG
└── 20250709-130603_DSC00003.JPG

One flat folder, holding photos and videos and nothing else. It sorts correctly two different ways, on purpose:

  1. The filename leads with the capture time, so a plain name sort is a chronological sort — even in tools that ignore file dates.
  2. The capture time is written onto the file itself, both Date Created and Date Modified, so sorting by date in Finder works too and every other app sees the real capture moment instead of the moment you copied it.

Sony's per-clip XML sidecars carry lens and GPS data that isn't in the MP4, so they're kept — in a hidden .sidecars/ folder you never see in Finder.

Install

Requires macOS 14+, Python 3.11+, and exiftool.

brew install exiftool
git clone https://github.com/ZechariahJ/SonyIngest.git
cd SonyIngest
uv tool install .          # or: pipx install .

Confirm everything is wired up:

sonyingest --self-check

The app

For one-click imports, build the little macOS app:

./app/build.sh --install

Double-click SonyIngest in your Applications folder. It finds the card, tells you how many files are new versus already imported, asks once, then shows a progress bar. Nothing is imported twice.

It's built locally rather than shipped as a download because it's signed ad-hoc — a downloaded copy would be blocked by Gatekeeper.

Usage

Look before you leap. --dry-run prints the complete plan and touches nothing:

sonyingest /Volumes/UNTITLED --dry-run

Then drop the flag:

sonyingest /Volumes/UNTITLED --dest ~/Pictures/Camera

Set a default destination once in ~/.config/sonyingest/config.toml and you can skip the flags entirely:

dest = "~/Pictures/Camera"
layout = "single"
rename = "prefix"
video_tz = "auto"
sidecar_xml = "hide"

Options

Flag Meaning
--dest PATH Library root. Default ~/Pictures/Camera, or dest from the config file.
--dry-run Print the full planned mapping, write nothing.
--layout single|flat|split single (default) puts everything in one folder. flat makes a folder per shooting day. split adds stills/ and video/ inside each day.
--rename prefix|keep prefix (default) gives YYYYMMDD-HHMMSS_ORIGINAL.EXT. keep leaves camera names alone.
--video-tz auto|utc|local|+HH:MM How to interpret QuickTime:CreateDate. See below.
--sidecar-xml hide|keep|discard hide (default) puts Sony's XML in .sidecars/. keep puts it beside the clip. discard skips it.
--move Delete sources after a verified copy. Also requires --i-know-what-im-doing.
--verify-duplicates Confirm each duplicate with a full checksum of both files. Slow — see below.
--eject Unmount the card after a fully verified ingest.
--open Reveal the library in Finder when done.
--json Machine-readable result, including a per-file tz_rule.
--notify macOS notification on completion.
--force Ingest a folder lacking the Sony DCIM/ + M4ROOT/ signature.

Subcommands

Command Meaning
sonyingest list-cards Print every mounted Sony card.
sonyingest relayout Reorganize an existing library into the current layout, in place, and re-stamp capture times. Moves rather than copies, so it's fast.
sonyingest self-check Verify dependencies and versions.
sonyingest install-agent Install a LaunchAgent that imports automatically on card insert.
sonyingest uninstall-agent Remove it.

The timezone problem

This is the part that quietly ruins naive ingest scripts, and the main reason this tool exists. On a real ZV-E10 card, a photo and a clip shot 29 seconds apart are stamped like this:

Source Value Frame
Still EXIF:DateTimeOriginal 2025:07:03 20:03:49 camera local
Clip sidecar <CreationDate> 2025-07-03T20:04:18-05:00 camera local + offset
Clip QuickTime:CreateDate 2025:07:04 01:04:18 UTC

Take QuickTime:CreateDate at face value and that clip files itself at 1am the next day. Convert it to your computer's local time instead and — whenever the camera's clock is set to a different zone than the computer, which is normal when travelling — it lands hours from the photos it was shot beside. Either way the interleaving breaks, which defeats the whole point.

So SonyIngest normalizes everything to a single frame: camera-local wall clock.

  1. StillsEXIF:DateTimeOriginal is already camera-local. Used as-is.
  2. Clips with a sidecar<CreationDate> is camera-local and states its own UTC offset. Trusted outright. This is the common path.
  3. Clips without a sidecarQuickTime:CreateDate is ambiguous, because some Sony bodies write UTC there and others write local time. Rather than guess, auto tests both readings against the file's modification time and keeps whichever lands closer. On exFAT the card's mtime string is itself camera-local wall clock, written moments after the clip closes, which makes it a sharp discriminator: the correct reading typically matches within seconds while the wrong one is off by the entire UTC offset.

The camera's own UTC offset is taken by majority vote from the sidecars on the card, falling back to the stills' OffsetTimeOriginal, then the system zone.

Every file records which rule fired. It shows up in --dry-run output and is stored as tz_rule in both the manifest and --json:

[4/6] copy  20250709-130641_C0004.MP4  444.0 MB  392.3 MB/s  [qt:utc->-05:00(auto,68s)]
[5/6] copy  20250712-231639_C0006.MP4   20.0 MB  396.4 MB/s  [xml:CreationDate]

Override the heuristic per run with --video-tz utc, local, or a fixed +HH:MM.

Duplicate detection

Nothing is copied to your machine twice. Before copying, each file on the card is matched against everything already in the library by content — not by name or location. So a file you renamed or moved inside the library is still recognized, and a file you deleted from the library comes back on the next import.

Matching uses an exact byte count plus a hash of eight evenly spaced 128 KiB samples, capped at 1 MiB read per file no matter how large it is. Re-checking a full 40 GB card takes about 35 seconds, against roughly 8 minutes if every file were fully checksummed.

The tradeoff is deliberate and worth stating plainly: two files could in principle share an exact byte count and identical content at all eight sampled offsets while differing somewhere in between. Two different camera captures never do this. Silently corrupted library files theoretically could, so --verify-duplicates full-hashes both sides when you want certainty, at the cost of re-reading the whole card.

Safety

The card holds the only copy of your footage, so the defaults are conservative.

  • The card is never written to. It's only ever opened for reading.
  • Every copy is verified. Each file is hashed while being read, then the written file is re-read and hashed again. Any mismatch is reported loudly and the run exits non-zero. This is precisely what Finder drag-and-drop doesn't do, and it's why cards silently corrupt on exFAT → APFS copies.
  • Copy, never move. --move requires a second explicit flag and only unlinks a source after its copy has verified.
  • Idempotent and resumable. Re-running skips what's already there. It never produces file (2).MP4.
  • Files with no readable timestamp go to unsorted/ rather than being given a guessed date. Zero-byte files are reported. Identically named files from different DCIM subfolders are disambiguated with a short content hash, not a counter.

A .sonyingest-manifest.jsonl at the library root records every ingested file: source, destination, hash, byte count, capture time, timezone rule, and timestamp.

Automatic import on card insert

sonyingest install-agent

Installs a LaunchAgent watching /Volumes. On any new mount it checks for the Sony folder signature and imports using your saved config, then posts a notification. Requires dest to be set in the config file. Remove it with sonyingest uninstall-agent. Logs go to ~/Library/Logs/sonyingest.log.

Development

uv run --with pytest --with blake3 pytest -q

The suite synthesizes a fake Sony card — correct folder layout, tiny but valid JPEG and MP4 stubs carrying real EXIF and QuickTime tags — so it runs without any hardware. tests/fixture.py is also usable standalone:

python3 tests/fixture.py ./fake-card

Coverage includes multi-day cards, RAW+JPEG pairing, missing sidecars, the UTC-vs-local decision for videos, hash-mismatch detection, re-run idempotency, --dry-run writing nothing, filename collisions across DCIM subfolders, duplicate detection surviving renames, and the sampled fingerprint's documented blind spot.

Compatibility

Developed and tested against a Sony ZV-E10 card containing 341 files (40 GB) spanning 22 shooting days, on macOS 15 / Apple Silicon.

The layout it targets — DCIM/ plus PRIVATE/M4ROOT/CLIP/ with Cxxxx M01.XML sidecars — is shared across the Sony Alpha line, so other bodies should work. MP_ROOT/ and .MXF are handled but untested. If you try it on another body, a report either way is welcome.

Contributing

Issues and pull requests are welcome. If you're reporting a timestamp problem, sonyingest /Volumes/YOURCARD --dry-run --json includes the tz_rule that fired for each file, which is usually enough to diagnose it.

License

MIT — see LICENSE.

About

Consolidate a Sony camera card's photos and clips into one chronological, checksum-verified library on macOS

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages