-
Notifications
You must be signed in to change notification settings - Fork 49
Speed up split object streaming #2791
Copy link
Copy link
Open
Labels
I3Minimal impactMinimal impactS4RoutineRoutineU4Nothing urgentNothing urgentenhancementImproving existing functionalityImproving existing functionalityneofs-storageStorage node application issuesStorage node application issuesperformanceMore of something per secondMore of something per second
Metadata
Metadata
Assignees
Labels
I3Minimal impactMinimal impactS4RoutineRoutineU4Nothing urgentNothing urgentenhancementImproving existing functionalityImproving existing functionalityneofs-storageStorage node application issuesStorage node application issuesperformanceMore of something per secondMore of something per second
Type
Fields
Give feedbackNo fields configured for issues without a type.
Is your feature request related to a problem? Please describe.
Node can try to collect more child objects at once for streaming to a client, not one by one.
Describe the solution you'd like
Think and try some alternatives to this code.
At least keeping 2 objects may be more effective. While streaming one object to a client, the second one is being fetched. This way, with the same network speed b/w nodes and clients, a client should not wait for a child object to be received from the next-door node. On the other hand, memory consumption will increase too.
Also, streaming objects directly looks fast, and memory independent. Changes to this code can be considered too. But we should track the progress and not stream any byte twice.
Describe alternatives you've considered
Keep it as is. At least, it works and we do not have yet issues about slow object assembling.