Add hermetic template recording and replay

[tobi]

Summary

Adds hermetic recording and deterministic replay for successful Liquid renders.

Liquid::TemplateRecorder.record("render.json") do
  Liquid::Template.parse(source).render!(assigns)
end

Liquid::TemplateRecorder.replay_from("render.json", mode: :verify).render

What is captured

• root template and every parsed partial, with SHA-256 integrity checks
• partial file contents while preserving the existing one-argument FileSystem API
• ordinary Hash/Array assigns without wrapping or changing render semantics
• only properties actually read from Liquid Drops (Drop ivars are never scraped)
• exact filter calls/results, engine strictness, and final output

Storage

• .json: an atomic session file containing every render in the block
• .jsonl: append-only, one compact and independently replayable render per line
  • file locking for concurrent writers
  • successful renders are durable independently
  • a truncated final line is recoverable
  • replay any record with index: (the last record is the default)

JSONL is deliberately the canonical production-sampling format rather than one large pretty JSON object. Compression is kept orthogonal: rotated JSONL can be compressed externally, and a future .jsonl.zst writer can use one independent zstd frame per record without changing the schema. A single long-lived compressed stream would make append/recovery/selection worse.

Replay modes

• compute: run available filters normally
• strict: return exact recorded filter results and reject a changed call sequence
• verify: compute normally and reject output differences

Design changes from the previous version

This is a ground-up simplification after reviewing the old implementation. It removes the global Template.parse monkey patch, proxy Hash/Array values, guessed Drop ivar extraction, lossy filter summaries, broken custom FileSystem keyword change, duplicated render wrappers, benchmark-specific CLI, and unrelated performance changes. Recording state is thread-local, nested sessions are rejected, JSON writes are atomic, and failures never delete a pre-existing destination.

Test plan

• bundle exec rake test: all lax/strict/strict2 unit and integration runs pass
• fixed the stale liquid-spec branch pin (the upstream PR is merged) and two current-main strict2/RuboCop test regressions so CI can install and run
• focused recorder suite covers Drop fidelity/privacy, nested Drops, includes and legacy file systems, class and instance parsing, multiple renders/templates, JSONL append/recovery/indexing, exact strict filter replay, checksum validation, failed writes, supported render argument forms, nesting, and concurrent sessions
• RuboCop: no offenses on changed Ruby files

See docs/template_recorder.md for the API and format tradeoffs.

[Copilot]

Pull Request Overview

This PR implements a comprehensive hermetic recording system for Liquid templates that enables perfect deterministic replay of template executions with massive size optimizations. The system captures complete template execution state - including all variable access patterns, filter calls, and file system interactions - and stores it in optimized JSON recordings for testing, debugging, and performance analysis.

Key changes:

• Implements hermetic template recording and replay with drop-free architecture
• Adds semantic key-based filter recording system for ~95% size reduction
• Provides TrackableHash/Array wrappers for comprehensive variable interaction capture

Reviewed Changes

Copilot reviewed 31 out of 33 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/unit/template_recorder_unit_test.rb	Comprehensive unit tests for template recorder functionality
test/unit/memory_file_system_unit_test.rb	Unit tests for in-memory file system component
test/unit/json_schema_unit_test.rb	Unit tests for JSON schema validation and serialization
test/unit/event_log_unit_test.rb	Unit tests for event logging and path parsing
test/unit/cli_roundtrip_test.rb	CLI integration tests for recording/replay workflow
test/unit/binding_tracker_unit_test.rb	Unit tests for object binding and loop context tracking
test/test_helper.rb	Minor parameter update for file system compatibility
test/integration/template_recorder_integration_test.rb	Integration tests for complex template scenarios
test/integration/tags/include_tag_test.rb	Updates file system interface for context parameter
test/integration/profiler_test.rb	Updates file system interface for context parameter
test/integration/error_handling_test.rb	Updates file system interface for context parameter
performance/theme_runner.rb	Refactored to support individual test execution and recording
performance/shopify/vision.database.yml	Database formatting improvements for better readability
performance/shopify/database.rb	Adds ProductDrop class for proper Liquid Drop object handling
performance/profile.rb	Enhanced profiling with better output and flame graph support
performance/memory_profile.rb	Updates to use refactored theme runner methods
performance/benchmark.rb	Updates to use refactored theme runner methods and YJIT detection
lib/liquid/template_recorder/*	Core recorder implementation with replayer, memory filesystem, and schema
lib/liquid/strainer_template.rb	Adds filter call recording hooks
lib/liquid/tags/for.rb	Adds loop event recording for proper iteration tracking
lib/liquid/partial_cache.rb	Updates file system interface for context parameter
lib/liquid/file_system.rb	Adds file read recording hooks and context parameter
lib/liquid/drop.rb	Adds drop property access recording hooks
lib/liquid.rb	Adds template recorder module require
RECORDER_IMPLEMENTATION_PLAN.md	Comprehensive implementation documentation

Comments suppressed due to low confidence (6)

test/unit/template_recorder_unit_test.rb:252

• This puts statement in a test method will produce output during test execution. Consider using a proper assertion or removing debug output.

    puts "Replayed #{found_items} out of #{expected_items.length} expected items" if found_items < expected_items.length

test/unit/template_recorder_unit_test.rb:423

• This puts statement in a test method will produce output during test execution. Consider using a proper assertion or removing debug output.

      puts "Root recorded as array due to loop recording behavior"

test/unit/template_recorder_unit_test.rb:499

• This puts statement in a test method will produce output during test execution. Consider using a proper assertion or removing debug output.

      puts "Items available for replay: #{root_vars.map { |item| item['name'] }.join(', ')}"

test/unit/template_recorder_unit_test.rb:522

• This puts statement in a test method will produce output during test execution. Consider using a proper assertion or removing debug output.

      puts "Verify mode failed as expected: #{e.message}"

test/integration/template_recorder_integration_test.rb:133

• This puts statement in a test method will produce output during test execution. Consider using a proper assertion or removing debug output.

      puts "Files captured: #{data['file_system'].keys}" unless files_captured

performance/theme_runner.rb:29

• [nitpick] The parameter name 'strictness' is unclear - consider a more descriptive name like 'strict_options' or 'strict_filters'.

    @strictness = strictness

[Copilot]

Version comparison using string equality may fail for semantic versioning. Consider using proper version comparison logic.

Suggested change

	recorded_version = @data['engine']['liquid_version']
	current_version = Liquid::VERSION
	recorded_version = Gem::Version.new(@data['engine']['liquid_version'])
	current_version = Gem::Version.new(Liquid::VERSION)

[Copilot]

The smart_merge method performs deep merging which could be expensive for large data structures. Consider optimizing for common cases or adding size limits.

Suggested change

	# Calculate the size of a nested data structure
	def calculate_size(data)
	case data
	when Hash
	data.sum { |_, v| calculate_size(v) } + data.size
	when Array
	data.sum { |v| calculate_size(v) } + data.size
	else
	1
	end
	end

[Copilot]

Creating a new Set for each call may be inefficient for deeply nested structures. Consider reusing the visited set or using a different circular reference detection approach.

Suggested change

	def self.ensure_serializable(obj, visited = Set.new)
	def self.ensure_serializable(obj, visited = nil)
	visited ||= Set.new

[Copilot]

The parse_path method is complex with multiple state variables and nested conditions. Consider breaking it into smaller methods or using a more structured parsing approach.

[isaacbowen]

:))))))))))))))))

[ianks]

This is really cool. I spent some time integrating this into our storefront to test it out, and here are some things that came up:

1. Multi-template architectures
Our storefront uses JSON templates that render multiple section templates in a single request. The current recorder design captures a single template source, but in practice we often have dozens of section templates rendering together. It might be helpful to support recording multiple templates in a single recording session, perhaps with a templates array that tracks each parsed template along with its name/path.

2. Template parsing flow variations
We noticed the recorder hooks into Liquid::Template.parse (class method), but in our codebase we create template instances first and then call parse on them:

template = Liquid::Template.new
template.parse(source, options)

The instance method interception works, but the RecordingTemplate wrapper approach might need some adjustments for this pattern.

3. Section rendering context
For applications using section-based architectures, it would be valuable if the recording could capture which sections were rendered and in what order. This would make it easier to trace execution flow in complex page layouts.

4. File path resolution
The file system recording works well. One thing we noticed is that the paths in our file system reads are often relative to different roots, so having some metadata about the base path context would help with replay accuracy.

5. Template source capture timing
When templates are parsed through adapter layers or caching mechanisms, the source might not be available at the Liquid::Template.parse call site. Consider capturing source at multiple points in the template lifecycle to handle these cases.

The JSON schema is well structured and the recorder's approach to tracking variable access and filter calls is solid. Would be happy to discuss extending this to handle multi-template scenarios if you're interested.