Skip to content

[FLINK-40239][runtime] Support additional Flink SQL regular expression functions in YAML Transform - #4491

Open
haruki-830 wants to merge 1 commit into
apache:masterfrom
haruki-830:FLINK-40239
Open

[FLINK-40239][runtime] Support additional Flink SQL regular expression functions in YAML Transform#4491
haruki-830 wants to merge 1 commit into
apache:masterfrom
haruki-830:FLINK-40239

Conversation

@haruki-830

Copy link
Copy Markdown
Contributor

Summary

This commit adds support for five additional Flink SQL regular expression functions in YAML Transform: REGEXP_EXTRACT, REGEXP_EXTRACT_ALL, REGEXP_COUNT, REGEXP_INSTR, and REGEXP_SUBSTR.

The functions use Java regular expression syntax and align with Flink SQL semantics for NULL values, invalid patterns, capture groups, and no-match cases.

Key Changes

  1. Additional Regular Expression Functions
  • Added implementations for the five missing regular expression functions.
  • Supports capture group indexes in REGEXP_EXTRACT and REGEXP_EXTRACT_ALL.
  • Uses the expected no-match behavior:
    • REGEXP_EXTRACT and REGEXP_SUBSTR return NULL.
    • REGEXP_EXTRACT_ALL returns an empty array.
    • REGEXP_COUNT and REGEXP_INSTR return 0.
  • Invalid regular expressions and invalid capture group indexes return NULL.
  1. YAML Transform Integration
  • Registered the new functions in the Transform SQL operator table.
  • Added return type inference for ARRAY<STRING> from REGEXP_EXTRACT_ALL.
  • Verified that REGEXP_EXTRACT_ALL works correctly in YAML Transform projections.
  • Existing REGEXP_REPLACE behavior remains unchanged.
  1. Test Coverage
  • Added unit tests covering NULL arguments, invalid patterns, no matches, capture groups, and invalid group indexes.
  • Added parser tests for function registration and expression translation.
  • Added projection tests covering the runtime conversion of REGEXP_EXTRACT_ALL results.
  1. Documentation
  • Documented the five new functions in both English and Chinese Transform documentation.

JIRA Reference

https://issues.apache.org/jira/browse/FLINK-40239

@github-actions github-actions Bot added docs Improvements or additions to documentation runtime labels Jul 27, 2026
@haruki-830
haruki-830 marked this pull request as ready for review July 28, 2026 06:32
@haruki-830

Copy link
Copy Markdown
Contributor Author

@lvyanquan @yuxiqian PTAL, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant