Refactor the dataset reader surface

[glopesdev]

The harp-data reader was shaped around callers holding a generated device package, where every register is reachable as a typed class. Someone with only a recorded folder had a harder path: addresses to look up, no way to see what a session actually contains, and a read_all that pulled in everything by default. This branch reshapes the surface so both audiences are served by the same reader, taking it from eight public members to six.

Removals are outright, with no deprecation shims, since the packages are pre-release and every downstream device package is regenerated on publication.

Construction

create_dataset_reader is replaced by open_dataset, which takes the folder first and an optional device module second, mirroring open_serial_device in the serial package.

reader = data.open_dataset("session.harp")            # builds the module from device.yml
reader = data.open_dataset("session.harp", behavior)  # module already in hand

schema and converters both describe how to build a module, so passing either beside one now raises TypeError rather than silently ignoring it. DatasetReader(module, root) remains public and constructible, exactly as Device is beside open_serial_device.

Reading and discovery

read accepts a register name alongside a class and an address. Names resolve through the device register map rather than the module namespace, which excludes the common registers, so read("WhoAmI") still reaches a dataset file that is on disk.

contents is the new starting point for an unfamiliar dataset. It maps register name to address for every register with data in the folder, keyed exactly as read accepts:

>>> reader.contents
{'WhoAmI': 0, 'TimestampSeconds': 8, 'DigitalInputs': 32, 'AnalogData': 33}

>>> frames = {name: reader.read(name) for name in reader.contents}

read_all is removed. Most devices log every register by default anyway, and the previous method also silently skipped files whose address the schema does not describe. The comprehension above replaces it and keeps the per-register options of read in reach.

Surface reduction

registers is gone, since it only repeated device_module.REGISTER_MAP. files becomes paths, which are the actual objects held by the collection, and is now documented as the physical debug view beside resolver. The default resolver now returns addresses in numeric order, where previously address 8 would sort after address 34 because the file names sort lexicographically.

Register repr

A register now renders as e.g. <AnalogData @33> rather than <class 'harp.device.behavior.AnalogData'>. This lives in harp-protocol and belongs on this branch because it is what keeps REGISTER_MAP readable once registers is gone from the reader.

The repr sits on the metaclass of RegisterBase, which is what reaches structured-payload registers such as OperationControl. Those derive from RegisterBase directly and never pass through the scalar metaclass, so putting it lower would cover only half the register set. Calling a register that already declares an address now raises TypeError instead of returning a copy at the new address, so RegisterU32(0x08) still declares one while WhoAmI(44) no longer clones it.

Empty buffer parsing

parse_to_dataframe would fail on any register with a sub-array payload field when given a buffer with no frames, because the column count was inferred from the data and numpy cannot infer a dimension from a zero-length array. It is now taken from the dtype, so an empty buffer returns an empty frame with the full column set. This affects AnalogData and Version in the test schema, 2 of 30 registers, so every register can now produce a correctly structured empty frame.

Registers with no data

A register declared in the device register map with no data in the folder now reads as an empty DataFrame carrying the same columns, rather than raising FileNotFoundError. The schema describes the structure of the data regardless of whether anything was recorded. Naming an absent chunk through suffix= still raises, since that is a mistake about the request rather than data that was never recorded.

read also takes timestamp as a bool defaulting to True, matching parse_to_dataframe, in place of a tri-state that inferred it from the payload-type bit of the first frame.

[bruno-f-cruz]

Looks good! One thing I realized but probably we can discuss later: the current design may allow us to return an empty, correctly structured (i.e. expected column names) dataframe if no file exists in the directory. I am not sure this is something we want, but this is probably one of the few packages I have seen around where the infrastructure affords it.

[glopesdev]

@bruno-f-cruz Agreed, I think we do want to return typed empty frames. The reason why generic loaders have to raise on a missing file is because a missing CSV says nothing about what the columns or the types should have been, so there is no empty result available to return.

In harp-data the register set comes from the folder device.yml or from the generated package, so the columns and the dtypes are known before anything is read. This puts us closer to a database where every table is declared and some happen to have no rows, and there the answer is an empty result set rather than an error.

Most of it is already in place, and I have pushed the one gap to this PR. parse_to_dataframe(cls, b"") was failing for registers with a sub-array payload field, AnalogData and Version in the test schema, because the column count was inferred from the data and numpy cannot infer a dimension from a zero-length array. Taking it from the dtype instead fixes it, so all 30 registers now return a correctly structured empty frame with the right column names and dtypes. What is left is only the decision in the reader about whether a missing file returns that frame or raises, which we can do in a separate PR.