Skip to content

Latest commit

 

History

History
603 lines (462 loc) · 21 KB

File metadata and controls

603 lines (462 loc) · 21 KB

Host Integration

This guide is for applications that already own the image/container side and want OpenMeta to handle metadata.

OpenMeta is not an image encoder. The usual pattern is:

  • decode metadata from one source file
  • query or edit it in MetaStore
  • hand prepared metadata to your own writer, encoder, or SDK

If you want the shortest end-to-end examples first, start with quick_start.md. For public API adoption status, see api_stability.md. For the stable flat host naming contract, see flat_host_mapping.md. For deterministic host compatibility baselines, see compatibility_dump.md. For generated XMP merge and writeback precedence, see xmp_sync_policy.md. For per-target writer preserve/replace guarantees, see writer_target_contract.md.

Pick The Integration Path

Use the narrowest public API that matches your host:

Host owns Use
Existing target file or template execute_prepared_transfer_file(...) + persist_prepared_transfer_file_result(...)
EXR writer build_exr_attribute_batch_from_file(...)
Host-owned metadata object model visit_metadata(...)
JPEG/JXL/WebP/PNG/JP2/BMFF encoder path prepare_metadata_for_target_file(...) + adapter view or backend emitter
Adobe DNG SDK objects/files dng_sdk_adapter.h

There is no public fuzzy-search API yet. Query through exact keys and build your own display or search layer on top.

Adapter Classes

OpenMeta splits host integration surfaces deliberately:

  • export-only naming/traversal surface: visit_metadata(...) for host-owned metadata mapping layers
  • export-only adapter: build_ocio_metadata_tree(...) for OCIO-style metadata trees
  • host-apply adapter: build_exr_attribute_batch(...) for EXR/OpenEXR header workflows
  • direct bridge: dng_sdk_adapter.h for applications that already use Adobe DNG SDK objects
  • narrow translator: libraw_adapter.h for orientation mapping into LibRaw flip space

1. Read Into MetaStore

#include "openmeta/simple_meta.h"

#include <array>
#include <cstddef>
#include <span>
#include <vector>

std::vector<std::byte> file_bytes = load_file_somehow("input.jpg");

openmeta::MetaStore store;
std::array<openmeta::ContainerBlockRef, 256> blocks {};
std::array<openmeta::ExifIfdRef, 512> ifds {};
std::vector<std::byte> payload(1 << 20);
std::array<uint32_t, 1024> payload_indices {};

openmeta::SimpleMetaDecodeOptions options;
openmeta::SimpleMetaResult result = openmeta::simple_meta_read(
    std::span<const std::byte>(file_bytes.data(), file_bytes.size()),
    store,
    blocks,
    ifds,
    payload,
    payload_indices,
    options);

store.finalize();

The caller owns the scratch buffers. That is deliberate: the API stays deterministic and easy to reuse in hot paths.

2. Query By Exact Key

#include "openmeta/meta_key.h"

openmeta::MetaKeyView key;
key.kind = openmeta::MetaKeyKind::ExifTag;
key.data.exif_tag.ifd = "ifd0";
key.data.exif_tag.tag = 0x010F;  // Make

for (openmeta::EntryId id : store.find_all(key)) {
    const openmeta::Entry& entry = store.entry(id);
    // Inspect entry.value and entry.origin.
}

This is the public lookup model today. If you need fuzzy search, substring search, or ExifTool-style display lookup, add that in your application layer.

3. Generic Host Metadata Traversal

Use the traversal API when your application owns the metadata object model and needs deterministic exported names plus the original Entry.

#include "openmeta/interop_export.h"

class MyMetadataSink final : public openmeta::MetadataSink {
public:
    void on_item(const openmeta::ExportItem& item) noexcept override
    {
        // Map item.name + item.entry into your host metadata object.
    }
};

openmeta::ExportOptions options;
options.style              = openmeta::ExportNameStyle::FlatHost;
options.name_policy        = openmeta::ExportNamePolicy::ExifToolAlias;
options.include_makernotes = true;

MyMetadataSink sink;
openmeta::visit_metadata(store, options, sink);

This keeps host-specific object ownership and write behavior outside OpenMeta.

4. Build An EXR Attribute Batch

This is the cleanest host-adapter path in OpenMeta today.

#include "openmeta/exr_adapter.h"

openmeta::ExrAdapterBatch batch;
openmeta::BuildExrAttributeBatchFileOptions options;

openmeta::BuildExrAttributeBatchFileResult result =
    openmeta::build_exr_attribute_batch_from_file(
        "source.jpg", &batch, options);

for (const openmeta::ExrAdapterAttribute& attr : batch.attributes) {
    // Forward attr.name, attr.type_name, and attr.value to your EXR writer.
}

OpenMeta does not need OpenEXR headers for this path. It exports a neutral batch of EXR-style attributes that your host can apply through OpenEXR or its own EXR writer.

5. Feed A Host-Owned JPEG Or JXL Encoder

There are two public patterns for encoder-owned output:

  • implement a backend emitter such as JpegTransferEmitter or JxlTransferEmitter
  • build an adapter view and consume one normalized list of operations

Adapter-View Pattern

Use this when you want one target-neutral operation list.

#include "openmeta/metadata_transfer.h"

class MySink final : public openmeta::TransferAdapterSink {
public:
    openmeta::TransferStatus
    emit_op(const openmeta::PreparedTransferAdapterOp& op,
            std::span<const std::byte> payload) noexcept override
    {
        // Dispatch on op.kind and forward payload into your backend.
        return openmeta::TransferStatus::Ok;
    }
};

openmeta::PrepareTransferFileOptions prepare;
prepare.prepare.target_format = openmeta::TransferTargetFormat::Jxl;

openmeta::PrepareTransferFileResult prepared =
    openmeta::prepare_metadata_for_target_file("source.jpg", prepare);

openmeta::PreparedTransferAdapterView view;
openmeta::build_prepared_transfer_adapter_view(
    prepared.bundle, &view, openmeta::EmitTransferOptions {});

MySink sink;
openmeta::emit_prepared_transfer_adapter_view(prepared.bundle, view, sink);

This is a good fit when your host already has its own abstraction for "metadata op + bytes".

Backend-Emitter Pattern

Use this when your host already looks like one OpenMeta backend.

#include "openmeta/metadata_transfer.h"

class MyJpegEmitter final : public openmeta::JpegTransferEmitter {
public:
    openmeta::TransferStatus
    write_app_marker(uint8_t marker_code,
                     std::span<const std::byte> payload) noexcept override
    {
        // Write one APPn marker into your JPEG output path.
        return openmeta::TransferStatus::Ok;
    }
};

openmeta::PrepareTransferFileOptions prepare;
prepare.prepare.target_format = openmeta::TransferTargetFormat::Jpeg;

openmeta::PrepareTransferFileResult prepared =
    openmeta::prepare_metadata_for_target_file("source.jpg", prepare);

openmeta::PreparedTransferExecutionPlan plan;
openmeta::compile_prepared_transfer_execution(
    prepared.bundle, openmeta::EmitTransferOptions {}, &plan);

MyJpegEmitter emitter;
openmeta::emit_prepared_transfer_compiled(prepared.bundle, plan, emitter);

For JXL, implement JxlTransferEmitter::set_icc_profile(...), add_box(...), and close_boxes(...).

OpenMeta does not ship a TurboJPEG-specific wrapper yet. The intended integration path is still through JpegTransferEmitter or the adapter view.

6. Edit An Existing Target File

If your host already has a target file or template on disk, use the file helper instead of building your own writer path.

#include "openmeta/metadata_transfer.h"

openmeta::ExecutePreparedTransferFileOptions exec_options;
exec_options.prepare.prepare.target_format =
    openmeta::TransferTargetFormat::Tiff;
exec_options.edit_target_path = "rendered.tif";

openmeta::ExecutePreparedTransferFileResult exec =
    openmeta::execute_prepared_transfer_file("source.jpg", exec_options);

openmeta::PersistPreparedTransferFileOptions persist;
persist.output_path = "rendered_with_meta.tif";
persist.overwrite_output = true;

openmeta::PersistPreparedTransferFileResult saved =
    openmeta::persist_prepared_transfer_file_result(exec, persist);

This path is usually simpler than a custom adapter when the container already exists.

Read Once, Save Later

If your host already decoded source metadata during the initial load, keep a decoded source snapshot and execute the later save without reopening the source file:

#include "openmeta/metadata_transfer.h"

const openmeta::ReadTransferSourceSnapshotFileResult snapshot =
    openmeta::read_transfer_source_snapshot_file("source.jpg");

openmeta::ExecutePreparedTransferSnapshotOptions options;
options.prepare.target_format = openmeta::TransferTargetFormat::Tiff;
options.edit_target_path      = "target.tif";
options.execute.edit_apply    = true;

openmeta::ExecutePreparedTransferFileResult result =
    openmeta::execute_prepared_transfer_snapshot(
        snapshot.snapshot, options);

Python mirrors that same host-facing snapshot flow:

from pathlib import Path

import openmeta

snapshot_info = openmeta.read_transfer_source_snapshot_file("source.jpg")
snapshot = snapshot_info["snapshot"]

result = openmeta.transfer_snapshot_file(
    snapshot,
    target_format=openmeta.TransferTargetFormat.Tiff,
    edit_target_path="target.tif",
    target_bytes=Path("target.tif").read_bytes(),
    output_path="edited.tif",
)

Current source snapshots are decoded-store-backed. They are intended for the normal EXIF/XMP/ICC/IPTC transfer flow, not raw source-packet passthrough. For hosts that still own the bundle/execution split, the lower-level prepare_metadata_for_target_snapshot(...) entry point remains available. If the host already has a decoded MetaStore, build a reusable snapshot with build_transfer_source_snapshot(store). If it already owns the source bytes in memory, use read_transfer_source_snapshot_bytes(bytes) instead of the file-path reader. In Python, a previously decoded Document can be turned into a reusable snapshot through doc.build_transfer_source_snapshot() or openmeta.build_transfer_source_snapshot(doc). If it also owns the destination bytes in memory, call the overload execute_prepared_transfer_snapshot(snapshot, target_bytes, options). If it already holds a prepared bundle, use execute_prepared_transfer_bundle(bundle, target_bytes, options) instead. Snapshot execution supports the same existing-sidecar merge and destination carrier-precedence controls as the file helper; when loading an existing sidecar it defaults to edit_target_path unless xmp_existing_sidecar_base_path is set explicitly. For embedded-only writeback with sidecar cleanup and no filesystem path, set xmp_existing_destination_sidecar_state explicitly so OpenMeta can return a cleanup decision without guessing a sidecar location. Python now exposes those same split path/state controls directly: xmp_existing_sidecar_base_path, xmp_sidecar_base_path, xmp_existing_destination_embedded_path, and xmp_existing_destination_sidecar_state.

The CLI and Python command-line wrapper do not implement their own transfer semantics. They map flags onto the same file-helper contract:

  • --output is the sidecar base for sidecar and embedded_and_sidecar writeback, so the generated sidecar is output-stem.xmp.
  • --xmp-writeback sidecar suppresses generated embedded XMP.
  • --xmp-writeback embedded_and_sidecar writes generated XMP to both the edited output and the generated sidecar.
  • embedded-only writeback preserves an existing destination sidecar unless --xmp-destination-sidecar strip_existing is selected.
  • sidecar-only writeback preserves existing destination embedded XMP unless --xmp-destination-embedded strip_existing is selected.
  • --force maps to the C++ persistence overwrite flags for the primary output and generated sidecar.

7. Query Runtime Capabilities

Hosts can ask OpenMeta what the current build supports before wiring format menus, warnings, or integration feature flags.

#include "openmeta/metadata_capabilities.h"

openmeta::MetadataCapability cap = openmeta::metadata_capability(
    openmeta::TransferTargetFormat::Avif,
    openmeta::MetadataCapabilityFamily::Xmp);

if (openmeta::metadata_capability_available(cap.target_edit)) {
    // The current build can edit AVIF XMP within the reported support level.
}

Each operation reports one of unsupported, supported, bounded, or disabled. bounded means the capability exists within OpenMeta's documented contract, not that it is arbitrary metadata-editor parity. disabled is used for compile-time-disabled support such as XMP decode when XML support is not available.

Python exposes the same query:

cap = openmeta.metadata_capability(
    openmeta.TransferTargetFormat.Avif,
    openmeta.MetadataCapabilityFamily.Xmp,
)
print(cap["target_edit_name"])

8. Use The Optional Adobe DNG SDK Bridge

If OpenMeta was built with OPENMETA_WITH_DNG_SDK_ADAPTER=ON, you can use the optional SDK bridge in two ways.

Update An Existing DNG File

#include "openmeta/dng_sdk_adapter.h"

openmeta::ApplyDngSdkMetadataFileResult result =
    openmeta::update_dng_sdk_file_from_file("source.jpg", "target.dng");

Apply Onto Existing SDK Objects

#include "openmeta/dng_sdk_adapter.h"
#include "openmeta/metadata_transfer.h"

openmeta::PrepareTransferFileOptions prepare;
prepare.prepare.target_format = openmeta::TransferTargetFormat::Dng;

openmeta::PrepareTransferFileResult prepared =
    openmeta::prepare_metadata_for_target_file("source.jpg", prepare);

openmeta::DngSdkAdapterOptions adapter;
openmeta::apply_prepared_dng_sdk_metadata(
    prepared.bundle, host, negative, adapter);

This bridge is for applications that already use the Adobe DNG SDK. OpenMeta still does not encode pixels or invent raw-image structure.

Host-Owned Image Specs

If a transfer target is produced from a different image buffer than the source, the host writer owns the target image facts: dimensions, channel count, sample type, compression, orientation, colorspace, ICC profile, and TIFF strip/tile storage. OpenMeta does not infer those values from copied metadata. During prepared transfer it filters source EXIF/XMP image-layout fields so stale source properties are not written into a different output image.

Host code that encodes pixels should keep those fields from the target container or inject values derived from the actual output buffer. Enable source ICC transfer only when the host has verified that the profile matches the target pixel buffer; otherwise preserve or write the target profile.

Use TransferProfile::safety for the broad source/destination relationship:

Mode Use when Transfer policy
CompatibleFile Metadata is repackaged or recompressed into a compatible file/pixel representation Preserve source camera, color, ICC, and camera-specific data except target-owned image-layout fields
RenderedImage Pixels may have changed, especially RAW-to-JPEG/PNG/WebP/JXL/HEIF/AVIF export Keep general/time/GPS/IPTC/portable XMP; drop source raw color calibration, linearization/crop/correction metadata, vendor RAW/source-processing geometry/color/correction/thermal/computational fields, camera raw settings XMP, source ICC, opaque MakerNotes, and non-C2PA JUMBF

See writer_target_contract.md for the detailed per-group transfer matrix.

openmeta::PrepareTransferRequest request;
request.target_format = openmeta::TransferTargetFormat::Jpeg;
request.profile.safety = openmeta::TransferSafetyMode::RenderedImage;

request.target_image_spec.has_dimensions = true;
request.target_image_spec.width = encoded_width;
request.target_image_spec.height = encoded_height;

request.target_image_spec.has_samples_per_pixel = true;
request.target_image_spec.samples_per_pixel = 3;
request.target_image_spec.bits_per_sample_count = 1;
request.target_image_spec.bits_per_sample[0] = 8;
request.target_image_spec.has_photometric_interpretation = true;
request.target_image_spec.photometric_interpretation = 2; // RGB
request.target_image_spec.has_exif_color_space = true;
request.target_image_spec.exif_color_space = 1; // sRGB

Python exposes the same structure as openmeta.TransferTargetImageSpec and the command-line wrappers pass it through without a separate policy layer:

spec = openmeta.TransferTargetImageSpec()
spec.has_dimensions = True
spec.width = encoded_width
spec.height = encoded_height
spec.has_samples_per_pixel = True
spec.samples_per_pixel = 3
spec.bits_per_sample = [8]
spec.has_photometric_interpretation = True
spec.photometric_interpretation = 2
spec.has_exif_color_space = True
spec.exif_color_space = 1

For smoke testing the file-helper path, metatransfer and python -m openmeta.python.metatransfer expose equivalent flags:

metatransfer --target-jpeg target.jpg -o output.jpg \
  --target-width 320 --target-height 240 \
  --target-samples-per-pixel 3 --target-bits-per-sample 8 \
  --target-photometric 2 --target-exif-color-space 1 \
  source.jpg

9. Query Phase One RAW Processing Metadata

After decoding MakerNotes, hosts can query Phase One/Leaf RAW processing data without depending on private MakerNote tag layout. The helper reports presence and normalized values for color matrices, WB RGB levels, black level, sensor temperatures, raw-data/storage byte counts, and sensor-calibration summaries. These values are source-RAW processing metadata; do not write them into rendered outputs unless the destination is a compatible RAW-style target.

#include "openmeta/phaseone_geometry.h"

openmeta::PhaseOneRawGeometryResult geometry =
    openmeta::phaseone_raw_geometry_from_store(store);
openmeta::PhaseOneRawProcessingResult raw =
    openmeta::phaseone_raw_processing_from_store(store);

if (raw.status == openmeta::PhaseOneRawProcessingStatus::Ok &&
    raw.info.has_color_matrix1) {
    const double m00 = raw.info.color_matrix1[0];
    (void)m00;
}

Python exposes the same normalized queries on decoded documents and reusable transfer snapshots:

doc = openmeta.read("source.iiq", decode_makernote=True)
geometry = doc.phaseone_raw_geometry()
raw = doc.phaseone_raw_processing()

if (raw["status"] == openmeta.PhaseOneRawProcessingStatus.Ok and
        raw["has_color_matrix1"]):
    m00 = raw["color_matrix1"][0]

The metaread command prints compact phaseone_raw_geometry=... and phaseone_raw_processing=... summaries when those decoded fields are present.

10. Query Vendor RAW Processing Metadata

For Sony, Canon, Nikon, Fujifilm, Pentax, Panasonic, Olympus, Kodak, Minolta, Sigma, Samsung, Ricoh, Apple, DJI, Google, and FLIR, OpenMeta exposes a conservative grouped summary instead of vendor-specific decoded values. The helper reports whether decoded MakerNote fields look like source RAW color/WB, geometry/storage, lens correction, raw-data, sensor-calibration, computational, thermal, or vendor-private table metadata. Use it to audit transfer safety decisions and host UI, not as a rendered-output write source.

#include "openmeta/vendor_raw_processing.h"

openmeta::VendorRawProcessingSummary sony =
    openmeta::vendor_raw_processing_from_store(
        store, openmeta::VendorRawProcessingFamily::Sony);

if (sony.fields_seen > 0) {
    const uint32_t unsafe_for_rendered = sony.color_fields +
        sony.white_balance_fields + sony.lens_correction_fields;
    (void)unsafe_for_rendered;
}

openmeta::TransferSafetyAudit audit =
    openmeta::transfer_safety_audit_from_store(
        store, openmeta::TransferSafetyMode::RenderedImage);

if (audit.filtered_raw_color_calibration > 0 ||
    audit.filtered_icc_profiles > 0 ||
    audit.filtered_makernotes > 0) {
    // Show the host/user which source-bound metadata will not be transferred.
}

Python uses the same family enum:

summary = doc.vendor_raw_processing(openmeta.VendorRawProcessingFamily.Nikon)
if summary["fields_seen"]:
    print(summary["lens_correction_fields"])

audit = doc.transfer_safety_audit(openmeta.TransferSafetyMode.RenderedImage)
print(audit["filtered_raw_color_calibration"])

metaread prints vendor_raw_processing[sony|canon|nikon|fujifilm|pentax|panasonic|olympus|kodak|minolta|sigma|samsung|ricoh]=... summaries when matching decoded fields are present.

11. Build MetaStore Yourself

If your application creates metadata directly, build the store first and then reuse the same export and transfer APIs.

#include "openmeta/meta_key.h"
#include "openmeta/meta_store.h"
#include "openmeta/meta_value.h"

openmeta::MetaStore store;
const openmeta::BlockId block = store.add_block(openmeta::BlockInfo {});

openmeta::Entry entry;
entry.key = openmeta::make_exif_tag_key(store.arena(), "ifd0", 0x010F);
entry.value = openmeta::make_text(
    store.arena(), "Vendor", openmeta::TextEncoding::Ascii);
entry.origin.block = block;
entry.origin.order_in_block = 0;

store.add_entry(entry);
store.finalize();

Related Docs