Skip to content

[Feature][Connector-V2] PR1: Pass sink table-options into auto-created MySQL target tables#11120

Open
luxiaolong-ct wants to merge 3 commits into
apache:devfrom
luxiaolong-ct:feature-11047-pr1
Open

[Feature][Connector-V2] PR1: Pass sink table-options into auto-created MySQL target tables#11120
luxiaolong-ct wants to merge 3 commits into
apache:devfrom
luxiaolong-ct:feature-11047-pr1

Conversation

@luxiaolong-ct

Copy link
Copy Markdown

Purpose of this pull request

Add table-options for JDBC sink auto-create flow (MySQL first delivery for #11047).

  • Introduce shared table-options config in SinkConnectorCommonOptions
  • Propagate options into CatalogTable in JdbcSinkFactory
  • Validate per dialect; MySQL/TiDB allow engine, charset, collate
  • Fail fast for unsupported JDBC dialects or unknown keys

Status — MySQL JDBC only; StarRocks/Paimon out of scope for this draft.

Does this PR introduce any user-facing change?

Yes. New optional sink config table-options (map). Applied only when schema_save_mode triggers DDL (e.g. CREATE_SCHEMA_WHEN_NOT_EXIST). Does not alter existing tables or runtime INSERT/UPSERT.

How was this patch tested?

  • Unit: MysqlDialectTest, MysqlCreateTableSqlBuilderTest
  • E2E: JdbcMysqlTableOptionsIT

Check list

@DanielLeens DanielLeens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. I reviewed the full diff from JdbcSinkFactory down into the MySQL create-table builder and the new E2E, rather than treating this as a docs-only option addition.

What this PR fixes

  • User pain: when JDBC sink auto-creates a MySQL target table, users currently have no real way to pass table-level options such as engine, charset, or collate through the sink config and get them into the generated DDL.
  • Fix approach: add shared table_options, propagate them into CatalogTable.options, validate them per dialect, and let the MySQL create-table builder append those clauses to the final CREATE TABLE.
  • One-line summary: this makes MySQL table-level auto-create options real on the actual DDL path instead of leaving them as unsupported config intent.

Runtime chain I checked

JDBC sink initialization
  -> JdbcSinkFactory.createSink(...) [connector-jdbc/.../JdbcSinkFactory.java:71-208]
      -> read table_options [73]
      -> put table_options into CatalogTable.options [183]
      -> load JDBC dialect [187-192]
      -> validateTableOptions(...) [193]

Dialect boundary
  -> JdbcDialect.validateTableOptions(...) [connector-jdbc/.../JdbcDialect.java:916-925]
      -> non-empty table_options fail fast by default
  -> MysqlDialect.validateTableOptions(...) [connector-jdbc/.../MysqlDialect.java:402-418]
      -> allow only engine / charset / collate

Auto-create path
  -> save-mode / catalog createTable(...)
      -> MysqlCreateTableSqlBuilder.build(...) [connector-jdbc/.../MysqlCreateTableSqlBuilder.java:138-157]
          -> ENGINE = ... [145-147]
          -> DEFAULT CHARSET = ... [148-150]
          -> COLLATE = ... [151-153]

TiDB compatibility boundary
  -> JdbcDialectLoader.load(url, compatibleMode, dialect, fieldIde)
      -> MySqlDialectFactory.create(...) [connector-jdbc/.../MySqlDialectFactory.java:44-48]
          -> still returns MysqlDialect for MySQL-family compatible modes except StarRocks

Key findings

  1. The normal MySQL auto-create path really does hit this change. table_options is not only accepted syntactically; it reaches the final DDL builder.
  2. The fail-fast behavior for unsupported dialects is the right safety boundary here. Unsupported JDBC dialects do not silently ignore user intent.
  3. I did not find a blocking correctness issue in the current head.
  4. The current docs mention MySQL / TiDB together. From the source path that is reasonable, because the sink still routes MySQL-family compatible modes through MysqlDialect; the current tests just do not add a TiDB runtime pass yet.

Test / stability note

  • MysqlDialectTest covers supported keys, unknown keys, and unsupported-dialect rejection.
  • MysqlCreateTableSqlBuilderTest checks that the generated DDL really contains the expected clauses.
  • JdbcMysqlTableOptionsIT uses SHOW CREATE TABLE plus row-count verification, which is the right signal for this feature and does not rely on weak timing assumptions. I did not see an obvious flaky-test anti-pattern in the new test code.

My merge conclusion for the current head is: can merge.

Blocking items

  • None from my side.

Suggested follow-up

  • A TiDB runtime coverage pass would still be useful later, since the docs explicitly mention MySQL / TiDB together and the current verification is source-level plus MySQL E2E only.

Overall, the implementation is clean and the scope is well controlled. The option, validation boundary, DDL wiring, E2E, and EN/ZH docs are aligned on the current head.

@davidzollo davidzollo added the First-time contributor First-time contributor label Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants