Skip to content

Goal: Duplicate record pruning #1414

Description

@vinarmani

Issue raised by trufnetwork/adapters#237

There are many streams on the network that consist of many duplicate entries. In some cases, the value of a stream has never changed from the first data point broadcast, yet there is a data point, with the same exact value, that was broadcast every 24 hours. There is no application where such repeat data points are necessary on the network.

  • In the case of a query for "current value," the current value is always the last point broadcast. For instance, although a market may not be open on a weekend, during the time that the market is closed, the last value broadcast (the closing price on Friday) is considered the "current price."
  • In the case of certified attestation or prediction protocol settlement, the attested value is the most recent value at the time of the request/settlement. It doesn't matter if that value was broadcast 1 month ago or one minute ago. Therefore, if the value has not changed in a month, the attestation of the month-old value is as valid as the attestation of the one-minute-old value.

In the same way that pruning of high-frequency streams is done, leaving only open-high-close values, pruning of repeat values older than a certain threshold (ie. 30 days), would provide significant reductions to resource requirements (storage) for network participants over time and at scale.

Deadline

ETA: 07-Sep-2026

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions