Skip to content

Speed up split object streaming #2791

@carpawell

Description

@carpawell

Is your feature request related to a problem? Please describe.

Node can try to collect more child objects at once for streaming to a client, not one by one.

Describe the solution you'd like

Think and try some alternatives to this code.
At least keeping 2 objects may be more effective. While streaming one object to a client, the second one is being fetched. This way, with the same network speed b/w nodes and clients, a client should not wait for a child object to be received from the next-door node. On the other hand, memory consumption will increase too.

Also, streaming objects directly looks fast, and memory independent. Changes to this code can be considered too. But we should track the progress and not stream any byte twice.

Describe alternatives you've considered

Keep it as is. At least, it works and we do not have yet issues about slow object assembling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    I3Minimal impactS4RoutineU4Nothing urgentenhancementImproving existing functionalityneofs-storageStorage node application issuesperformanceMore of something per second

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions