Exporting every single combination of gamemode, rank, style/ability EVERY DAY is not very scalable. It may have performed fine in the beginning, when there were only a few thousand lines of data, but for every season, it grows. There is now over 540 thousand lines of data, and it shows on vbl.gg. What used to load instantly, now takes an entire second to display on my Galaxy S25, due to how the data is filtered and analyzed client-side.
How can this be improved?
Stop exporting data for every single day
While being able to combine data for multiple dates can be useful, it dramatically increases the data size. Even if data was exported once every two days, it would half the data size, while still allowing for good analyzation.
Reduce the amount of combinations exported (remove casual)
With how many combinations are possible for gamemodes, ranks and styles/abilities, the data grows exponentially when a new style or ability is added. I would argue that there is no real benefit of including data for the casual gamemodes. When analyzing these files, there is little to no interest in knowing which styles/abilities perform better in casual mode.
Smaller optimizations
- Use dates only (yyyy-mm-dd) instead of a full ISO-timestamp.
- This saves 14 bytes of data per line
- "T00:00:00.000Z" contains no useful information, and is currently included in every line
- Shorten names, either by ID or acronyms
- "OnesMini" could be reduced to "1M"
- "Twos_Pro_Style_Encho" could be reduced to "2_PSEncho"
- Current ranks all start with a different letter (B, S, G, D, P, L, M, C, U)
- S/A = Style/Ability (technically not required)
- I assume keeping the style/ability names is your interest, so the website can at least display them in the default rarity
I've also attached the custom code I use for analyzing the statistics. It requires Deno
vbl-stats.zip
If you wish to discuss this, you can either reply here, or find me in the programs server (wommy)
Exporting every single combination of gamemode, rank, style/ability EVERY DAY is not very scalable. It may have performed fine in the beginning, when there were only a few thousand lines of data, but for every season, it grows. There is now over 540 thousand lines of data, and it shows on vbl.gg. What used to load instantly, now takes an entire second to display on my Galaxy S25, due to how the data is filtered and analyzed client-side.
How can this be improved?
Stop exporting data for every single day
While being able to combine data for multiple dates can be useful, it dramatically increases the data size. Even if data was exported once every two days, it would half the data size, while still allowing for good analyzation.
Reduce the amount of combinations exported (remove casual)
With how many combinations are possible for gamemodes, ranks and styles/abilities, the data grows exponentially when a new style or ability is added. I would argue that there is no real benefit of including data for the casual gamemodes. When analyzing these files, there is little to no interest in knowing which styles/abilities perform better in casual mode.
Smaller optimizations
I've also attached the custom code I use for analyzing the statistics. It requires Deno
vbl-stats.zip
If you wish to discuss this, you can either reply here, or find me in the programs server (wommy)