perf: cache repeated long string rendering

[He-Pin]

Motivation:

This is stacked after #856 (perf/simple-format-ascii-safe). The benchmark comparison below is
against origin/perf/simple-format-ascii-safe@f2a2e3ba, where this PR adds one commit:
7656943f perf: cache repeated long string rendering.

kube-prometheus renders many repeated medium/long description strings. Without caching, each repeated value pays the full quoted-string rendering cost again: UTF-8 encoding, JSON escape scanning, and byte copying. This showed up as a remaining Native renderer gap after larger parser/import and inline-object sort work had already been addressed.

Key Design Decision:

Cache only fully rendered quoted bytes for repeated strings in the 128..4096 character range, and cap the cache by both entry count and rendered byte size. This keeps the optimization targeted at repeated Kubernetes-style descriptions while leaving huge unique strings, such as large_string_template, on the existing renderer path.

Modification:

• Add a per-renderer HashMap[String, Array[Byte]] in BaseByteRenderer.
• Cache rendered quoted bytes only when the source string has 128..4096 characters.
• Cap cached rendered entries at 16 KiB each and at 2048 entries per renderer.
• Copy cached bytes directly into the renderer byte buffer on repeated hits.
• Add a renderer regression test covering repeated long escaped strings.
• Update the performance ledgers with fresh JMH and Native hyperfine evidence.

Benchmark Results:

Environment:

• Branch base: origin/perf/simple-format-ascii-safe@f2a2e3ba
• Candidate: perf/long-string-render-cache@7656943f
• JVM: JDK 21.0.10, JMH 1.37
• Native binaries:
  • base: /tmp/sjsonnet-native-long-cache-pr-base
  • candidate: /tmp/sjsonnet-native-long-cache-pr-candidate
• Benchmark rule: single benchmark process; no concurrent Mill/JMH/hyperfine.

Output equality:

workload	check
kube-prometheus realworld	base output equals candidate output
bench/resources/cpp_suite/large_string_template.jsonnet	base output equals candidate output

JMH guard:

Command:

./mill --no-server --ticker false --color false bench.runRegressions bench/resources/cpp_suite/large_string_template.jsonnet

variant	score
base	0.666 ms/op
candidate	0.666 ms/op

Native hyperfine: kube-prometheus, forward order, 12 runs:

variant	mean	stddev	median	min	max
base	132.889 ms	2.539 ms	132.431 ms	128.896 ms	137.725 ms
candidate	132.280 ms	2.789 ms	131.247 ms	129.262 ms	138.730 ms

Native hyperfine: kube-prometheus, reverse order, 12 runs:

variant	mean	stddev	median	min	max
candidate	130.720 ms	1.562 ms	130.347 ms	128.133 ms	133.569 ms
base	132.262 ms	1.457 ms	132.585 ms	130.240 ms	134.842 ms

Native hyperfine: large_string_template, forward order, 30 runs:

variant	mean	stddev	median	min	max
base	11.074 ms	1.293 ms	11.126 ms	8.941 ms	13.997 ms
candidate	10.822 ms	0.931 ms	10.738 ms	9.237 ms	13.076 ms

Native hyperfine: large_string_template, reverse order, 30 runs:

variant	mean	stddev	median	min	max
candidate	11.266 ms	1.065 ms	10.977 ms	9.805 ms	14.192 ms
base	11.781 ms	1.040 ms	11.805 ms	10.201 ms	14.117 ms

Validation:

./mill --no-server --ticker false --color false __.checkFormat
./mill --no-server --ticker false --color false -j 1 __.test

Result:

Tests: 445, Passed: 445, Failed: 0

Analysis:

The kube-prometheus target is positive in both command orders, which is the expected workload for this cache because it contains repeated rendered description strings. large_string_template remains neutral-to-positive in both command orders because its dominant string is huge and unique, so it is intentionally above the cache cap and stays on the existing fast path. The JMH guard is exactly neutral at 0.666 ms/op, so this does not trade Native improvement for a JVM regression.

References:

• bench/reports/sjsonnet-vs-jrsonnet-gaps.md
• bench/reports/sync-points.md

Result:

Repeated medium/long strings render faster on kube-prometheus with bounded memory overhead, unchanged output, neutral JVM guard behavior, and full test-suite coverage.