Skip to content

Add JSON column type support#209

Merged
PNixx merged 2 commits intoPNixx:masterfrom
DeweyLabs:json-data-type
Nov 17, 2025
Merged

Add JSON column type support#209
PNixx merged 2 commits intoPNixx:masterfrom
DeweyLabs:json-data-type

Conversation

@ScotterC
Copy link
Copy Markdown
Contributor

Summary

This PR adds support for ClickHouse's JSON column type to fix Rails schema dump failures that occur with "Unknown type 'JSON' for column" errors.

Changes Made

  • Core Implementation: Added json: { name: 'JSON' } to NATIVE_DATABASE_TYPES constant
  • Type Mapping: Registered JSON type mapping using ActiveRecord's built-in Type::Json class
  • Comprehensive Tests: Added 7 new test cases covering:
    • JSON column type recognition and validation
    • CRUD operations with JSON data
    • Complex JSON structure handling
    • Migration support with t.json syntax
    • Integration with insert_all operations

Technical Details

  • Uses ActiveRecord's standard Type::Json for consistent behavior
  • Handles ClickHouse JSON type limitations appropriately:
    • JSON columns cannot be nullable (ClickHouse limitation)
    • Numbers are stored/retrieved as strings (ClickHouse behavior)
  • Requires allow_experimental_json_type = 1 setting (ClickHouse 21.1+)

Testing

  • ✅ All 7 new JSON-specific tests pass
  • ✅ All 85 existing single-server tests pass (no regressions)
  • ✅ All 6 cluster tests pass (no regressions)
  • ✅ Manual verification confirms type support works

Usage Example

# Migration
class AddJsonColumns < ActiveRecord::Migration[7.1]
  def change
    create_table :products, request_settings: { allow_experimental_json_type: 1 } do |t|
      t.string :name
      t.json :properties, null: false
      t.json :metadata, null: false
    end
  end
end

# Model Usage
Product.create!(
  name: "Widget", 
  properties: { "color" => "red", "size" => "large" },
  metadata: { "tags" => ["electronics", "gadget"] }
)

Compatibility

  • ClickHouse: 21.1+ (JSON type introduced)
  • Rails: 7.1+ (existing gem requirement)
  • Experimental Feature: Requires allow_experimental_json_type = 1

This minimal, backwards-compatible implementation resolves schema dump issues while providing full JSON column support for ClickHouse users.

🤖 Generated with Claude Code

@PNixx
Copy link
Copy Markdown
Owner

PNixx commented Sep 5, 2025

@ScotterC testing failed

Adds support for ClickHouse JSON column type to fix schema dump failures.

- Add json: { name: 'JSON' } to NATIVE_DATABASE_TYPES constant
- Register JSON type mapping using ActiveRecord::Type::Json
- Add comprehensive test coverage for JSON column operations
- Include tests for column recognition, CRUD operations, and migrations
- Handle ClickHouse JSON type limitations (non-nullable, numbers as strings)

Resolves "Unknown type 'JSON' for column" errors in Rails schema dumps.
Requires allow_experimental_json_type = 1 setting in ClickHouse 21.1+.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Use MergeTree engine for JSON migration test instead of relying on the
default Log engine. The Log engine doesn't support JSON columns with
dynamic subcolumns in ClickHouse, which causes test failures in version
25.10.

This change ensures compatibility with both ClickHouse 24.9 and 25.10.

All 98 tests now pass on both versions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@ScotterC
Copy link
Copy Markdown
Contributor Author

@PNixx updated to fix specs

@PNixx PNixx merged commit 2b19fd7 into PNixx:master Nov 17, 2025
20 checks passed
woodhull added a commit to daisychainapp/clickhouse-activerecord that referenced this pull request Nov 24, 2025
This merge brings in the latest improvements from upstream including:
- JSON column type support (PNixx#209)
- Fixed regex to match FROM keyword (PNixx#220)
- Active Record 8.1 support
- SQL structure dumper improvements
- Better format logic encapsulation (PNixx#162)
- Schema dumper fixes (PNixx#206)
- Unscope :final and :settings (PNixx#208)

Fork-specific functionality preserved:
- EOFError and Errno::ECONNRESET retry logic (integrated into raw_execute)
- Custom insert settings via ClickhouseActiverecord::Settings thread-local
- disconnect! method implementation
- Security fix: no hardcoded OpenSSL::SSL::VERIFY_NONE
- Async insert configuration support

The retry logic has been adapted to work with the new execute/raw_execute
architecture introduced by upstream, ensuring compatibility with the refactored
code structure while maintaining reliability for connection errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants