Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for binlog_row_image NOBLOB #1257

Open
altmannmarcelo opened this issue May 16, 2024 · 0 comments
Open

Add support for binlog_row_image NOBLOB #1257

altmannmarcelo opened this issue May 16, 2024 · 0 comments

Comments

@altmannmarcelo
Copy link
Contributor

Description

MySQL has the ability to reduce the size of the binlog transmitted between source and replica by sending a minimal row image. This helps to reduce the network bandwidth between servers.

Minimal/noblob row differs from full rows as follow:

   - INSERTS

      |-----------+--------------+-----------------------------------|
      | OPTION    | BEFORE IMAGE | AFTER IMAGE                       |
      |-----------+--------------+-----------------------------------|
      | 'minimal' | ---          | All columns where a value         |
      |           |              | was specified. Autoinc columns    |
      |           |              | are also set if not specified.    |
      |-----------+--------------+-----------------------------------|
      | 'noblob'  | ---          | All columns where a value         |
      |           |              | was specified, and all            |
      |           |              | non-blob columns. Autoinc columns |
      |           |              | are also set if not specified.    |
      |-----------+--------------+-----------------------------------|
      | 'full     | ---          | All columns                       |
      |-----------+--------------+-----------------------------------|

      - UPDATES

      |-----------+----------------------------+---------------------------|
      | OPTION    | BEFORE IMAGE               | AFTER IMAGE               |
      |-----------+----------------------------+---------------------------|
      | 'minimal' | PKE                        | All columns where a value |
      |           |                            | was specified             |
      |-----------+----------------------------+---------------------------|
      | 'noblob'  | PKE + all non-blob columns | All columns where a value |
      |           |                            | was specified, and all    |
      |           |                            | non-blob columns          |
      |-----------+----------------------------+---------------------------|
      | 'full     | All columns                | All columns               |
      |-----------+----------------------------+---------------------------|

      - DELETES
 
      |-----------+----------------------------+-------------|
      | OPTION    | BEFORE IMAGE               | AFTER IMAGE |
      |-----------+----------------------------+-------------|
      | 'minimal' | PKE                        | ---         |
      |-----------+----------------------------+-------------|
      | 'noblob'  | PKE + all non-blob columns | ---         |
      |           |                            |             |
      |           |                            |             |
      |-----------+----------------------------+-------------|
      | 'full'    | All columns                | ---         |
      |-----------+----------------------------+-------------|

https://dev.mysql.com/worklog/task/?id=5092

With noblob the before image will have the PK or UK + all non-blob fields. We need to either need the full row (which will not be send if the table has blobs) or identify the PKE (which will be mixed with all non-blob columns in the before row).

In order to support noblob we should have update and delete TableOperation that take into consideration that the key will contain the columns for the keys + some data. This new operations should accept that key.len() has more fields that the actual base table key. and using a mapping we should index the column position and its value (this information is available via TableMapEvent and BinlogRow) and extract the columns that are PK.

Change in user-visible behavior

Requires documentation change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant