You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
We want to allow projection pushdown of compute inside aggr_expr into lower nodes, ideally a table scan. Currently, this is done for Projection nodes, but not for aggregates with compute inside them.
Aggregate: groupBy=[[hits.CounterID]], aggr=[[avg(__aggregate_arg_1) AS avg(octet_length(hits.URL)), count(Int64(1))]]
Projection: hits.CounterID, CAST(octet_length(hits.URL) AS Float64) AS __aggregate_arg_1
SubqueryAlias: hits
Filter: hits_raw.URL != Utf8View("")
TableScan: hits_raw projection=[CounterID, URL], partial_filters=[hits_raw.URL != Utf8View("")]
This is a cost based change, but as a start its always beneficial to do this if that expr is the only consumer of a column [keeping the scheme len non increasing].
Then existing TableScan can try and accept the projection expr. If the table scan doesn't accept this then there is not extra compute.
We cannot do this if the Agg contains a local Filter.
Is your feature request related to a problem or challenge?
We want to allow projection pushdown of compute inside
aggr_exprinto lower nodes, ideally a table scan. Currently, this is done forProjectionnodes, but not for aggregates with compute inside them.Consider
Describe the solution you'd like
The possible solution I would like is to extract each the scalar compute into a Projection.
Before:
After:
This is a cost based change, but as a start its always beneficial to do this if that expr is the only consumer of a column [keeping the scheme len non increasing].
Then existing
TableScancan try and accept the projection expr. If the table scan doesn't accept this then there is not extra compute.We cannot do this if the Agg contains a local
Filter.Describe alternatives you've considered
placementto pushdownbit/octet/''_length#25025Additional context
Us at vortex would like to compute complex projection pushdown over vortex-compressed data