Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fixed merge between int64 uint64 and wrote test #58332

Closed
wants to merge 1 commit into from

Conversation

YKim0172
Copy link

When you have two DataFrames, left and right, where left's key is of dtype int64 while right's key is of dtype uint64, and both of their values are >= 2**53, what used to happen was that calling left.merge(right) will cause a non empty result to be returned. However, we want the merge function to return nothing since we shouldn't be allowing the merge between DataFrames with different key dtypes.

The issue was that when you are working with values >= 253 and you have keys that are of dtype int64 and uint64, the "common dtype" gets resolved to float64 for both, so a (253) int64 and (2**53 + 1) uint64 is represented as the same float64 value.

@YKim0172 YKim0172 changed the title Fixed merge between int64 uint64 and wrote test BUG: Fixed merge between int64 uint64 and wrote test Apr 21, 2024
Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in #56367 (comment), this appears to already work so it would just need a test

@mroeschke mroeschke added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Apr 22, 2024
@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Merges erroneously match integer join keys between int64 and uint64 when using values >= 2**53
2 participants