You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
Currently we save indices to the batch always as u64 in the HashTable and in the nextVec.
If we have less than u32:MAX (4.2B, i.e. most of the time) items in the build sidre, we can store them as u32 - which should make it fit more easily in the CPU cache.
Describe the solution you'd like
Implement optimization to store indices as u32 if possible.
Run benchmarks
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
@Dandandan For this, is the preferable solution to create a generic parameter, and when the hash join exec/ stream is created, based on the size of the build side we can assign the generic that value? or an enum could also possibly work? What do you think?
@Dandandan For this, is the preferable solution to create a generic parameter, and when the hash join exec/ stream is created, based on the size of the build side we can assign the generic that value? or an enum could also possibly work? What do you think?
I think we'll likely have to:
add a generic type to JoinHashMap<T>
make a Box<dyn JoinHashMap> based on the num_rows (num_rows <= u32::MAX => u32 else u64)
Uh oh!
There was an error while loading. Please reload this page.
Is your feature request related to a problem or challenge?
Currently we save indices to the batch always as
u64
in theHashTable
and in thenext
Vec
.If we have less than
u32:MAX
(4.2B, i.e. most of the time) items in the build sidre, we can store them asu32
- which should make it fit more easily in the CPU cache.Describe the solution you'd like
u32
if possible.Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: