Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support constant array. #5701

Open
liurenjie1024 opened this issue Apr 29, 2024 · 6 comments
Open

Support constant array. #5701

liurenjie1024 opened this issue Apr 29, 2024 · 6 comments
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@liurenjie1024
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

A constant array would be useful for improving performance, see apache/iceberg-rust#295 (comment)

Describe the solution you'd like

Allow constructing constant array.

Describe alternatives you've considered

Additional context

@tustvold
Copy link
Contributor

Is https://docs.rs/arrow-array/latest/arrow_array/struct.Scalar.html perhaps what you are looking for?

@liurenjie1024
Copy link
Contributor Author

Is https://docs.rs/arrow-array/latest/arrow_array/struct.Scalar.html perhaps what you are looking for?

Not quite same. Here is the case, when we want to convert iceberg predicate to ArrowPredicateFn, say a < 1, for some reason here a is a constant, e.g. NULL, we can always return true. But ArrowPredicateFn requires to return an BooleanArray.

@mapleFU
Copy link
Member

mapleFU commented Apr 30, 2024

https://arrow.apache.org/docs/format/Columnar.html

Seems arrow doesn't have an constant array ( But array with type "Null" is similiar to that), maybe send this to maillist is prefered?

@liurenjie1024
Copy link
Contributor Author

https://arrow.apache.org/docs/format/Columnar.html

Seems arrow doesn't have an constant array ( But array with type "Null" is similiar to that), maybe send this to maillist is prefered?

Cool, I will send a mail to dev list.

@tustvold
Copy link
Contributor

Constant propogation is something I'd expect to be handled by a query optimizer, perhaps in conjunction with range analysis. FWIW Datafusion is already capable of this

@liurenjie1024
Copy link
Contributor Author

Yes, it possible to be handled by expression rewrite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

3 participants