fix(ast): replace DescribeStatement stubs with UnsupportedStatement, fix extraction gaps

[ajitpratap0]

Summary

Comprehensive quality sweep addressing architectural, QA, and DX issues identified in the v1.14.0 project review. This PR touches 20 files across the AST, parser, formatter, and gosqlx packages.

P0 — Critical

• New UnsupportedStatement AST node: Replaces misuse of DescribeStatement for Snowflake stubs (USE, COPY INTO, PUT, GET, LIST, REMOVE, CREATE STAGE/STREAM/TASK/PIPE/etc). Consumers doing switch stmt.(type) can now distinguish these from real DESCRIBE statements. Includes pool (Get/Put), TokenLiteral(), and Children().
• EXTRACT parser: Added parseExtractExpression() for EXTRACT(YEAR FROM expr) syntax — was completely missing, causing parse failures.
• 7 extraction gaps fixed: All t.Skip() stubs in extract_test.go replaced with real passing tests for CASE, CAST, IN, BETWEEN, EXTRACT, and recursive CTEs.

P1 — High

• AST.HasUnsupportedStatements(): New method to detect stub statements in parsed output.
• Formatter handles unsupported statements: Emits -- UNSUPPORTED: {RawSQL} comment instead of producing corrupt SQL (e.g., DESCRIBE COPY for COPY INTO).
• Coverage test audit: Removed stale comments, added TODO(v2-cleanup) markers to 5 overlapping parser coverage files (4,286 lines across 51 test functions).

P2 — Medium

• Validate() reconciled: parser.ValidateBytes now rejects empty input, matching gosqlx.Validate behavior.
• ParseBytes fixed: Threads []byte directly to tokenizer instead of []byte→string→[]byte round-trip.
• pkg/sql/monitor deprecated: Added deprecation notice pointing to pkg/metrics, targeting v2.0 removal.
• Deprecation timeline: All 3 legacy parser APIs (Parse([]token.Token), ParseFromModelTokensWithPositions, ConversionResult.PositionMapping) now have "Scheduled for removal in v2.0".

Changes

File	What changed
pkg/sql/ast/ast.go	New UnsupportedStatement type + AST.HasUnsupportedStatements()
pkg/sql/ast/pool.go	Pool, ReleaseStatement case, Get/Put for UnsupportedStatement
pkg/sql/ast/pool_ddl_test.go	Pool tests for UnsupportedStatement
pkg/sql/parser/parser.go	parseSnowflakeUseStatement and parseSnowflakeStageStatement now produce UnsupportedStatement; deprecation timeline on 3 APIs
pkg/sql/parser/ddl.go	12 Snowflake CREATE stubs now produce UnsupportedStatement with captured raw SQL
pkg/sql/parser/expressions_complex.go	New parseExtractExpression()
pkg/sql/parser/expressions_literal.go	EXTRACT dispatch before generic identifier handler
pkg/sql/parser/validate.go	Empty-input rejection
pkg/sql/parser/validate_test.go	Updated expectation for empty input
pkg/formatter/render.go	UnsupportedStatement case in FormatStatement
pkg/formatter/render_ddl.go	renderUnsupported() — emits SQL comment
pkg/gosqlx/gosqlx.go	ParseBytes threads bytes directly
pkg/gosqlx/extract.go	Removed "Known Limitations" doc (all fixed)
pkg/gosqlx/extract_test.go	7 t.Skip() stubs → real passing tests
pkg/sql/monitor/doc.go	Deprecation notice
5 coverage test files	TODO(v2-cleanup) markers

Test Plan

• go build ./... — clean
• go vet ./... — clean
• go test -race -timeout 120s ./... — all packages pass, zero failures
• Pre-commit hooks pass (fmt, vet, tests)
• All 7 previously-skipped extract tests now pass
• EXTRACT parsing verified: SELECT EXTRACT(YEAR FROM created_at) FROM orders
• Snowflake stubs produce UnsupportedStatement instead of DescribeStatement
• Formatter produces -- UNSUPPORTED: ... comments for stub statements
• Validate("") returns error consistently at both API levels
• Existing DescribeStatement usage for real MySQL DESCRIBE/DESC unchanged

Breaking Changes

• parser.ValidateBytes("") now returns an error instead of nil. This matches gosqlx.Validate("") which already rejected empty input. Callers that relied on empty-input-is-valid should add an explicit check before calling.
• Snowflake stub statements changed type: *ast.DescribeStatement → *ast.UnsupportedStatement. Code doing stmt.(*ast.DescribeStatement) on Snowflake USE/COPY/PUT/GET/etc. will need updating. The DescribeStatement type is preserved for actual MySQL DESCRIBE/DESC/EXPLAIN commands.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
gosqlx-website	Ready	Preview, Comment	Apr 12, 2026 8:09am

[github-actions]

Claude finished @ajitpratap0's task —— View job

---

Code Review for PR #511

Reviewing this comprehensive quality sweep that addresses architectural, QA, and DX issues. Let me analyze the changes across the 20 files.

Todo List:

• Review AST changes and UnsupportedStatement implementation
• Analyze parser improvements for EXTRACT functionality
• Check formatter handling of unsupported statements
• Evaluate test coverage improvements
• Assess breaking changes and migration impact
• Review performance and security considerations
• Provide final recommendations

🟢 Excellent Implementation Quality

AST Architecture

• UnsupportedStatement design: Clean, focused node with Kind and RawSQL fields pkg/sql/ast/ast.go:1947-1954
• Pool integration: Proper Get/Put functions pkg/sql/ast/pool.go:130-134, 1767-1782
• Type safety: Correct interface implementations with statementNode(), TokenLiteral(), Children()
• AST utility: HasUnsupportedStatements() method for detection pkg/sql/ast/ast.go:1898-1905

Parser Improvements

• EXTRACT implementation: Complete parser function parseExtractExpression() pkg/sql/parser/expressions_complex.go:394-430
• Smart integration: Added to primary expression parsing dispatch pkg/sql/parser/expressions_literal.go:125-129
• SQL standard compliance: Proper EXTRACT(field FROM source) syntax

Formatter Excellence

• UnsupportedStatement handling: renderUnsupported() produces safe SQL comments pkg/formatter/render_ddl.go:125-130
• Prevents corruption: -- UNSUPPORTED: {RawSQL} instead of malformed output
• Integration: Proper case handling in main dispatcher pkg/formatter/render.go:167-168

Test Coverage Improvements

• All skipped tests fixed: 7 t.Skip() stubs → real passing tests
• Comprehensive scenarios: CASE, CAST, IN, BETWEEN, EXTRACT, recursive CTEs
• Quality assurance: Proper test structure with setup/teardown

🟡 Breaking Changes - Well Handled

1. ValidateBytes Empty Input (Minor)

// Before: ValidateBytes("") returned nil
// After: ValidateBytes("") returns error

Impact: Low - Empty SQL should fail validation
Rationale: Aligns with gosqlx.Validate("") behavior

2. Snowflake Statement Types (Major)

// Before: USE statement → *ast.DescribeStatement 
// After: USE statement → *ast.UnsupportedStatement

Impact: High for Snowflake users doing type assertions
Migration: Clear in PR description with examples

🟢 Security & Performance

• Memory safety: All pooling patterns correctly implemented
• No security issues: No credentials, injection vectors, or unsafe operations
• Performance neutral: Object pooling maintained, no regressions

🟢 Architecture Benefits

1. Type safety: Eliminates DescribeStatement misuse for Snowflake stubs
2. Future-proof: UnsupportedStatement provides extensible pattern
3. Developer experience: Clear distinction between real/stub statements
4. Formatter safety: Prevents corrupt SQL generation

Minor Observations

1. Line 41 in validate.go: trimBytes(input) check is good for empty input validation
2. DDL stubs: Snowflake CREATE STAGE/STREAM/etc. properly handled in pkg/sql/parser/ddl.go:156-159

Recommendation: APPROVE ✅

This is exemplary work:

• Addresses real architectural debt
• Maintains backward compatibility where possible
• Documents breaking changes clearly
• Comprehensive test coverage
• Security and performance conscious
• Clean, maintainable code

The PR successfully transforms architectural stubs into production-ready functionality while maintaining the library's performance characteristics.

---