Read OPC attribute layers from per-vertex aara data, show them under the 3D cursor - #669
Conversation
Newer OPC exports ship their attribute layers twice: as texture layers under
Images/<Layer>/, and additionally as per-vertex *.aara grids inside each patch
directory, listed in patch.xml's <Attributes>. Reading the per-vertex form needs
three small random-access reads per layer instead of a full image decode, which
makes attribute extraction fast enough to run interactively.
VertexAttributes reads those grids and samples them barycentrically at a picked
point. The attribute grid is *smaller* than the position grid - the positions carry
a symmetric skirt whose width shrinks with the hierarchy level (1032/1028/1026
against a constant 1024) - so the mapping is
attributeIndex(x, y) = positionIndex(x + off, y + off), off = (posSize - attrSize) / 2
verified for every patch of the Dimorphos export by matching LonLatRad's radius
channel against |Local2Global * XYZ_Local| to float32 round-off.
Profile extraction prefers this path and falls back to texture sampling only for
layers it does not cover. Three problems in the existing texture path came out of
comparing the two:
- texture layers store values normalised into the layer's ChannelsDefinedRange
while per-vertex layers hold physical values, so the profile CSV was exporting
[0,1] numbers and nodata (-99999) as a real value. Samples are now mapped back
onto the layer's range and nodata is dropped.
- the layer loop counted DiffuseColorNWeights entries as textures and capped at
numTextures - 1, so it only ever reached 3 of 8 layers.
- buildPatchInfoLookup keyed patches case-sensitively, but Patches_DirAbsPath
probes "patches" before "Patches" while persisted kd-trees may carry either, so
extraction silently found nothing on affected datasets.
The 3D preview cursor is on by default and its readout is the "Under Cursor"
section of the Config panel. It uses per-vertex data only - the texture fallback
cannot run per mouse move. Extraction happens on the existing background picking
thread.
Multi-channel ChannelsDefinedRange/ChannelsActualRange now parse; they previously
threw, so OPCs with vector-valued layers needed a hand-edited *.opcx to import at
all. The first channel's range is kept, matching what the false-colour legend and
the de-normalisation above refer to.
The triangle-to-grid mapping is stored as one int per quad rather than an int[3]
per triangle - 4 MB instead of ~100 MB per 1032^2 patch - and its cache is bounded,
because the cursor pulls in a new patch whenever it crosses a patch boundary.
OpcMetadata reads the *.opc.json sidecar: product provenance, the DEM reference
model (DemSphere's axis, DemEllipsoid's frame and radii) and, for DSK-derived OPCs,
the DSKBRIEF summary of the source *.bds shape model. Logged at import, not
persisted into the scene.
Closes #668
Review of #669 found the triangle-to-grid mapping wrong in a way that produced plausible but displaced values. The object set that hit.SetObject.Index refers to is built by DebugKdTreesX.loadTriangles' = PRo3DCSharp.ComputeIndexArray + getTriangleSet, and that path does NOT compact. An invalid quad leaves six zero indices, so it becomes the degenerate triangle (positions[0], positions[0], positions[0]) - which the NaN filter only drops when positions[0] is itself NaN. The mapping enumerated valid quads only, so on a patch with a valid vertex 0 and any invalid quads every lookup was displaced by the number of invalid quads before the hit. The Dimorphos patches this feature targets have an all-NaN border row, so positions[0] is NaN, the fillers are dropped and both orders coincide - which is why the radius invariant passed. A patch with interior holes and a valid corner would have read values from the wrong vertices instead. computeTriangleQuadStarts now mirrors that predicate: valid quads get their index, invalid ones get -1 when the fillers survive and are skipped when they do not. TryGetTriangleIndices returns None for -1. Two tests in TriangleSetTests pin both cases against the actual ComputeIndexArray + getTriangleSet output; the first fails without the fix. Also from the review: - buildPatchInfoLookup mutated an unsynchronized Dictionary now reached from both the background picking thread and profile export on the update thread. Locked, and clearCaches drops it too - it was left behind despite the doc claiming otherwise. - VertexAttributes used two locks for three pieces of state, and the eviction path cleared layerCache while holding only the fileCache lock. One lock now guards fileCache, layerCache and the byte accounting. - The "already covered" set compared layer names case-sensitively. Texture names come from the Images/<Layer> folder and per-vertex ones from the *.aara base name, so a spelling difference let the less accurate texture value through and, in extractProfile, override the per-vertex one. Compared case-insensitively now. - attributeIndex silently accepted grid size differences it cannot centre: an odd difference means an asymmetric skirt and a negative one a larger attribute grid, both of which shift every sample by a vertex. Such a layer is refused with a warning and falls back to texture sampling. - A preview pick with no hit left the previous point's values on screen as if current; the read-out is cleared now. Its empty-state wording also claimed the surface has no per-vertex layers when the real cause can be a hit on the skirt.
|
Reviewed with a sub-agent. It found one real defect in the load-bearing assumption, which is now fixed and pinned by tests, plus four smaller ones. All in The mapping followed the wrong triangle numbering
The mapping enumerated valid quads only. So on a patch with a valid vertex 0 and any invalid quads, every lookup was displaced by the number of invalid quads preceding the hit — reading elevations and slopes off the wrong vertices, plausibly and silently. It passed the radius invariant because the Dimorphos patches have an all-NaN border row, so
Also fixed
Not acted on
Solution builds; |
|
Follow-up filed as #670. |
Closes #668. Related: #667 (drop PRo3D's duplicate opcx parser), aardvark-platform/OPCViewer.Base#2 (same parser bug upstream).
Newer OPC exports ship their attribute layers twice: as texture layers under
Images/<Layer>/, and additionally as per-vertex*.aaragrids in each patch directory. Reading the per-vertex form needs three small random-access reads per layer instead of a full image decode, which makes attribute extraction fast enough to run interactively.Docs:
docs/VertexAttributes.md.The grid offset
The attribute grid is smaller than the position grid — the positions carry a symmetric skirt whose width shrinks with the hierarchy level, because it is a constant number of source DEM pixels:
Established empirically:
LonLatRad's third channel is the vertex radius in metres, so it must equal|Local2Global · XYZ_Local|. Across all 9 patches ofg_01960mm_spc_dtm_dimo_0000n00000_v003the centred offset agrees to float32 round-off (median 2·10⁻⁶ m); every other offset is off by ≥ 0.19 m. The same invariant guards the code as a test.Three problems the comparison surfaced in the existing texture path
ChannelsDefinedRange; per-vertex layers hold physical values. The profile CSV was therefore exporting[0,1]numbers, and the nodata sentinel-99999as if it were a measurement. Samples are now mapped back onto the layer's range and nodata is dropped.DiffuseColorNWeightsentries as textures and capped atnumTextures - 1, so it only ever reached 3 of the 8 layers on this dataset.buildPatchInfoLookupkeyed on the full positions path case-sensitively, butOpcPaths.Patches_DirAbsPathprobes"patches"before"Patches"(andDirectory.Existsis case-insensitive on Windows) while persisted kd-trees may carry either spelling. Now case-insensitive with separator normalisation.Under Cursor readout
The 3D preview cursor is on by default and its readout is the Under Cursor section of the Config panel, below Screenshots. It shows the surface, patch and every per-vertex layer under the mouse (hold
CTRL).Per-vertex data only — the texture fallback is one image decode per layer and cannot run per mouse move, so a surface without per-vertex layers says so rather than stalling. Extraction runs on the existing background picking thread.
Memory
The triangle-to-grid mapping is now one int per quad instead of an
int[3]per triangle — 4 MB instead of ~100 MB per 1032² patch — and the cache is bounded to 32 patches, because the cursor pulls in a new patch whenever it crosses a patch boundary. Caches are dropped when a surface is removed. ZIP-backed OPCs cannot seek, so their payloads are held in memory instead, bounded at 512 MB.Multi-channel
*.opcxrangesFor multi-channel
Maplayers ExportGpc writes one range per channel:Range1d.Parseonly understands[min, max], so these OPCs could not be imported without hand-editing the*.opcx. Both forms parse now. The first channel's range is kept — that is the channel the false-colour legend and the de-normalisation above refer to (attribute textures are read viaChannelWithIndex 0); unioning the channels would widen theGravityrange by 25% and skew every de-normalised value.*.opc.jsonsidecarOpcMetadatareads the sidecar next to the*.opcxand logs it at import (not persisted into the scene): product provenance, the DEM reference model, and for DSK-derived OPCs the DSKBRIEF summary of the source*.bds.DemSphereandDemEllipsoidhave different schemas (singleAxisvsAxisX/Y/Z+Radii) and both are read.Verification
Built and run against
C:\pro3ddata\HERA\OPCUpdate:OpcSidecar10/10,ProfileAttributeExtraction2/2 (4 skipped — they needDimorphos_DRACO1, absent here). Data-backed tests self-skip; point them withPRO3D_AARA_OPC/PRO3D_BDS_OPC.*.opcx: fed the original ExportGpc file, the parser yields ranges identical to the hand-patched one, layer for layer. A copy of the OPC with the original sidecar sits next to the patched one for end-to-end checks.7 per-vertex layers (LonLatRad, Normal, Gravity, Magnitude, Potential, Elevation, Slope), no lookup warnings,[OpcMetadata]block logged for both theDemSphereandDemEllipsoidexports.Not verified: the Under Cursor panel has not been screenshotted, only exercised by hand.