Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 38 additions & 30 deletions BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ Comprehensive performance comparison between all json2xml implementations.
## Test Environment

- **Machine**: Apple Silicon (arm64)
- **OS**: macOS 26.4.1 (Darwin 25.4.0)
- **OS**: macOS 26.5 (Darwin 25.5.0)
- **Python**: 3.14.4
- **Date**: April 24, 2026
- **Date**: May 27, 2026
- **CLI tools**: `json2xml-go` and `json2xml-zig` from `/Users/vinitkumar/.local/bin`

To make new runs comparable, record the same fields for your machine before
publishing results:
Expand All @@ -33,65 +34,72 @@ which json2xml-go json2xml-zig 2>/dev/null || true
| Size | Description | Bytes |
|------|-------------|-------|
| Small | Simple object `{"name": "John", "age": 30, "city": "New York"}` | 47 |
| Medium | 10 generated records with nested structures | 3,212 |
| Medium | 10 generated records with nested structures | 3,215 |
| bigexample.json | Real-world patent data | 2,018 |
| Large | 100 generated records with nested structures | 32,207 |
| Very Large | 1,000 generated records with nested structures | 323,148 |
| Large | 100 generated records with nested structures | 32,206 |
| Very Large | 1,000 generated records with nested structures | 323,131 |

## Results

### Performance Summary

| Test Case | Python | Rust | Go | Zig |
|-----------|--------|------|-----|-----|
| Small (47B) | 31.49µs | 0.55µs | 4.09ms | 2.02ms |
| Medium (3.2KB) | 1.69ms | 16.15µs | 4.07ms | 2.09ms |
| bigexample (2KB) | 819.86µs | 6.44µs | 4.37ms | 2.11ms |
| Large (32KB) | 17.97ms | 168.21µs | 4.10ms | 2.42ms |
| Very Large (323KB) | 183.33ms | 1.42ms | 4.20ms | 5.12ms |
| Small (47B) | 3.19µs | 0.86µs | 6.05ms | 3.08ms |
| Medium (3.2KB) | 214.83µs | 18.41µs | 5.85ms | 3.12ms |
| bigexample (2KB) | 91.20µs | 7.32µs | 5.76ms | 3.08ms |
| Large (32KB) | 2.07ms | 175.46µs | 5.89ms | 3.73ms |
| Very Large (323KB) | 21.20ms | 1.48ms | 6.82ms | 7.82ms |

### Speedup vs Pure Python

| Test Case | Rust | Go | Zig |
|-----------|------|-----|-----|
| Small (47B) | **56.8x** | 0.0x* | 0.0x* |
| Medium (3.2KB) | **105.0x** | 0.4x* | 0.8x* |
| bigexample (2KB) | **127.2x** | 0.2x* | 0.4x* |
| Large (32KB) | **106.8x** | 4.4x | **7.4x** |
| Very Large (323KB) | **129.0x** | **43.6x** | **35.8x** |
| Small (47B) | **3.7x** | 0.0x* | 0.0x* |
| Medium (3.2KB) | **11.7x** | 0.0x* | 0.1x* |
| bigexample (2KB) | **12.5x** | 0.0x* | 0.0x* |
| Large (32KB) | **11.8x** | 0.4x* | 0.6x* |
| Very Large (323KB) | **14.4x** | **3.1x** | **2.7x** |

*CLI tools have process spawn overhead (~2-4ms) which dominates for small inputs.
*CLI tools have process spawn overhead (~3-6ms) which dominates for small inputs.

## Key Observations

### 1. Rust Extension is the Best Choice for Python Users 🦀

The Rust extension (json2xml-rs) provides:
- **~57-129x faster** conversion than pure Python in this run
- **~4-14x faster** conversion than the optimized pure Python path in this run
- **Zero process overhead** - called directly from Python
- **Automatic fallback** - pure Python used if Rust is unavailable or a feature requires it
- **Easy install**: `pip install json2xml[fast]`

### 2. Go Excels for Very Large CLI Workloads 🚀
### 2. Python Optimizations Changed the Relative Speedups

Recent pure Python improvements substantially reduced conversion time:
- Small inputs dropped to microsecond-level library-call timings
- Medium and large inputs are roughly an order of magnitude faster than the April 2026 run
- Rust remains the fastest library backend, but its relative speedup is lower because the baseline improved

### 3. Go Excels for Very Large CLI Workloads 🚀

For very large inputs (323KB+):
- **43.6x faster** than Python
- But ~4ms startup overhead hurts small file performance
- **3.1x faster** than optimized Python in this run
- But ~6ms startup overhead hurts small and medium file performance
- Best for batch processing or large file conversions from shell scripts

### 3. Zig is Highly Competitive for CLI Use ⚡
### 4. Zig is Highly Competitive for CLI Use ⚡

In this run:
- **35.8x faster** than Python for very large files
- **7.4x faster** for large files (32KB)
- Faster startup than Go (~2ms vs ~4ms)
- **2.7x faster** than optimized Python for very large files
- Slower than optimized Python for the 32KB library-call benchmark once process startup is included
- Faster startup than Go (~3ms vs ~6ms)
- Best balance of startup time and throughput for mixed CLI workloads

### 4. Process Spawn Overhead Matters
### 5. Process Spawn Overhead Matters

CLI tools (Go, Zig) have process spawn overhead:
- Go: ~4ms startup overhead
- Zig: ~2ms startup overhead
- Go: ~6ms startup overhead
- Zig: ~3ms startup overhead
- Dominates for small inputs (makes them appear slower than Python)
- Negligible for large inputs where actual work dominates
- Rust extension avoids this entirely by being a native Python module
Expand All @@ -100,9 +108,9 @@ CLI tools (Go, Zig) have process spawn overhead:

| Use Case | Recommended | Why |
|----------|-------------|-----|
| Python library calls | **Rust** (`pip install json2xml[fast]`) | 57-129x faster, no process overhead |
| Small files via CLI | **Zig** (json2xml-zig) | Fastest startup among native CLIs (~2ms) |
| Large files via CLI | **Go** or **Zig** | Both excellent; Zig wins at 32KB, Go wins at 323KB in this run |
| Python library calls | **Rust** (`pip install json2xml[fast]`) | 4-14x faster, no process overhead |
| Small files via CLI | **Zig** (json2xml-zig) | Fastest startup among native CLIs (~3ms) |
| Large files via CLI | **Go** or **Zig** | Both excellent; Go wins at 323KB in this run |
| Batch processing | **Go** or **Rust** | Both excellent depending on shell vs Python integration |
| Pure Python required | **Python** (json2xml) | Always available |

Expand Down
Loading
Loading