Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expression: fix the collation of functions with json arguments #53126

Merged
merged 1 commit into from May 11, 2024

Conversation

YangKeao
Copy link
Member

@YangKeao YangKeao commented May 9, 2024

What problem does this PR solve?

Issue Number: close #52833

Problem Summary:

MySQL will use binary collation when the max_length of some arguments' types are too long. It'll mostly happen on JSON types.

What changed and how does it work?

As TiDB doesn't have a max_length attribute for each type and "datum", I specially handled the situation for JSON. Some function will return binary collation when some of the arguments are JSON type.

In MySQL, the state of BINARY flag, BLOB type and binary collation is really a chaos. For this example, you'll find that sometimes the collation(...) returns utf8mb4_bin while the returned type info (shown through --column-type-info) tells you the collation is utf8mb4_0900_ai_ci. I'm not sure whether it's a bug for MySQL or not...

So, I want to fix the behavior with minimum modification: only change the collation to binary collation and don't touch the charset.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Fix the issue that the collation of the result of some functions are not compatible with MySQL when the arguments contain JSON.

Copy link

codecov bot commented May 9, 2024

Codecov Report

Attention: Patch coverage is 97.43590% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 74.5693%. Comparing base (788d505) to head (f90772c).
Report is 30 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #53126        +/-   ##
================================================
+ Coverage   74.4098%   74.5693%   +0.1595%     
================================================
  Files          1490       1492         +2     
  Lines        356051     436989     +80938     
================================================
+ Hits         264937     325860     +60923     
- Misses        71835      91088     +19253     
- Partials      19279      20041       +762     
Flag Coverage Δ
integration 49.2414% <82.0512%> (?)
unit 71.2585% <92.3076%> (-1.9139%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9957% <ø> (-2.3014%) ⬇️
parser ∅ <ø> (∅)
br 50.0550% <ø> (+7.1665%) ⬆️

@YangKeao YangKeao force-pushed the fix-52833 branch 2 times, most recently from dc1e98e to cdce63c Compare May 9, 2024 08:03
@YangKeao
Copy link
Member Author

YangKeao commented May 9, 2024

/retest

1 similar comment
@YangKeao
Copy link
Member Author

YangKeao commented May 9, 2024

/retest

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>
@YangKeao
Copy link
Member Author

/retest

@lcwangchao lcwangchao added compatibility-breaker Violation of forwards/backwards compatibility in a design-time piece. and removed needs-1-more-lgtm labels May 11, 2024
Copy link

ti-chi-bot bot commented May 11, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lcwangchao, XuHuaiyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label May 11, 2024
Copy link

ti-chi-bot bot commented May 11, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-05-11 04:46:40.952235368 +0000 UTC m=+1282954.709370939: ☑️ agreed by lcwangchao.
  • 2024-05-11 08:33:32.403930292 +0000 UTC m=+1296566.161065860: ☑️ agreed by XuHuaiyu.

@YangKeao
Copy link
Member Author

/retest

@YangKeao
Copy link
Member Author

/retest

1 similar comment
@YangKeao
Copy link
Member Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit dcd1fa9 into pingcap:master May 11, 2024
23 checks passed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #53199.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-6.1: #53200.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request May 11, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-6.5: #53201.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request May 11, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request May 11, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.1: #53202.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #53203.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved compatibility-breaker Violation of forwards/backwards compatibility in a design-time piece. lgtm release-note size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

incorrect result when inlining CTE
4 participants