Skip to content

[Feature][Transform-V2] Add Base64 SQL functions#11114

Merged
davidzollo merged 1 commit into
apache:devfrom
MyeoungDev:feature/sql-transform-base64-functions
Jun 19, 2026
Merged

[Feature][Transform-V2] Add Base64 SQL functions#11114
davidzollo merged 1 commit into
apache:devfrom
MyeoungDev:feature/sql-transform-base64-functions

Conversation

@MyeoungDev

@MyeoungDev MyeoungDev commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Purpose of this pull request

Add Base64 conversion functions to SQLTransform.

  • TO_BASE64(value)
  • TO_BASE64(value, charset)
  • FROM_BASE64(value)
  • FROM_BASE64(value, charset)

The default charset is UTF-8. TO_BASE64(value) also supports byte[] input.

Closes #11107

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@DanielLeens DanielLeens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. I reviewed the latest head from the actual SQL transform execution path, not just from the function registration diff.

What this PR fixes

  • User pain: SQL transform already has functions like RAWTOHEX / HEXTORAW, but it still lacks a direct Base64 encode/decode pair for normal SQL expressions.
  • Fix approach: the PR registers TO_BASE64 / FROM_BASE64, wires their return type through ZetaSQLType, implements the runtime logic in StringFunction, and adds unit tests, SQL-level tests, E2E coverage, and docs.
  • One-line summary: this is a clean SQL string-function expansion that fills a real usability gap in the current function matrix.

Actual runtime path

SQLTransform execution
  -> ZetaSQLFunction dispatch [ZetaSQLFunction.java:475-480]
      -> TO_BASE64 / FROM_BASE64
          -> StringFunction.toBase64 / fromBase64 [StringFunction.java:192-233]
              -> string input
                  -> EncodingUtils.tryParseCharset(...)
                  -> charset-aware Base64 encode/decode
              -> byte[] input
                  -> TO_BASE64 encodes raw bytes directly
                  -> bytes + charset is rejected explicitly
  -> ZetaSQLType marks both functions as STRING [ZetaSQLType.java:327-334]

Review result

  • I did not find a source-level blocker on the current head.
  • The implementation is consistent with the existing SQL function architecture and stays nicely scoped.
  • The important edge cases are covered: byte-array input, optional charset, invalid charset, invalid Base64 content, SQL-level invocation, and E2E output assertions.
  • Compatibility looks good because this only adds new callable functions; existing queries do not change behavior.

CI note

  • Build is still red, but the failing jobs currently shown are unit-test (8, windows-latest), jdbc-connectors-it-part-1, and connector-file-sftp-it.
  • That failure set is much broader than the SQL transform files touched here. I could confirm the job names from the run metadata, but GitHub's log stream kept getting cancelled in this environment, so I cannot fully prove from logs yet that all three are unrelated. From the code path alone, I do not see a reason for this PR to break JDBC connector IT or SFTP connector IT.

Conclusion: can merge

Blocking items (must fix)

  • No source-level blocking item from Daniel on the current head.
  • The remaining gate is CI only: the contributor/maintainer still needs to clear the red Build checks before GitHub can merge.

Suggested follow-up (non-blocking)

  • No additional non-blocking suggestion from my side.

Overall assessment

  • This is a well-scoped feature PR with solid test layering and matching docs.
  • The runtime path is straightforward, and the new functions should be reached normally in SQL transform execution.

@nzw921rx

Copy link
Copy Markdown
Collaborator

+1 if CI pass

@DanielLeens

Copy link
Copy Markdown
Contributor

Thanks for checking. I agree with that read: from my side the source path on the current head looks good, so the remaining gate is just the CI result.

@davidzollo davidzollo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
LGTM
Thanks for your contribution! This is a solid addition to the project, and your attention to detail really shows. Contributors like you are what keep this community moving forward. Feel free to keep the PRs coming — we're always glad to have you here!

@davidzollo davidzollo merged commit 3f88dc9 into apache:dev Jun 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Transform-V2] SQLTransform support Base64 encode/decode functions

4 participants