Skip to content

How to stucture aggregated files? #4

Description

@mhidas

The "natural" way to store measured values from multiple profiles is in a two-dimensional array, with a profile index, and a pressure/level index, as is done in the core-Argo format. In the general case where profiles have different numbers of levels, the CF conventions call this incomplete multidimensional array representation. It is simple to read data from such files, but they can be large and contain lots of fill values (because their size is set by the profile with the maximum number of levels). The extra space taken up by the fill values is largely eliminated when the file is compressed, though this comes at the cost slightly increased access time.

The alternative is the contiguous ragged array representation, which packs all the profiles "end-to-end" into a single dimension, removing the need for fill values. Reading values from this structure is slightly more complex because of the need to keep track of the length of each profile.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions