Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,117 @@ $.person.*~

---

## Spectral Compatibility

Spectral rulesets use a safe JavaScript-flavored JSONPath subset that is not
part of RFC 9535. Enable it explicitly when executing Spectral `given`
selectors:

```go
import (
"github.com/pb33f/jsonpath/pkg/jsonpath"
"github.com/pb33f/jsonpath/pkg/jsonpath/config"
)

path, err := jsonpath.NewPath(
`$.paths[?(@property && !@property.match(/\/openapi\.json/))]~`,
config.WithSpectralCompatibility(),
)
```

`WithSpectralCompatibility()` implies the property-name extension and all
JSONPath Plus features, including context variables, `~`, and `^`.
`WithPropertyNameExtension()` may still be supplied and is redundant but
valid. `WithLazyContextTracking()` is independent and produces the same
results in eager and lazy modes.

Spectral compatibility and `WithStrictRFC9535()` are mutually exclusive.
Supplying both returns a configuration error regardless of option order. The
default dialect remains unchanged.

### Compatibility matrix

| Status | Feature | Behavior |
|--------|---------|----------|
| Supported | Truthiness | Missing values, `null`, `false`, zero, and empty strings are false; arrays, objects, non-empty strings, and non-zero numbers are true |
| Supported | Regex literals | `/pattern/flags`, compiled once with Go's linear-time RE2 engine |
| Supported | `value.match(regex)` | Partial regex search on strings; usable only as a truthy test |
| Supported | `value.indexOf(string[, fromIndex])` | JavaScript UTF-16 code-unit indexing on strings |
| Supported | `value.includes(value[, fromIndex])` | Safe membership checks on strings and sequences |
| Supported | `value.length` | UTF-16 length for strings and element count for sequences |
| Supported | `value.constructor.name` | Read-only synthetic type metadata: `Array`, `Object`, `String`, `Number`, or `Boolean` |
| Supported | `void 0` | The missing/undefined value used by official Spectral selectors |
| Supported | `===` and `!==` | JavaScript strict equality, including reference identity for arrays and objects |
| Supported | `==` and `!=` | JavaScript scalar coercion and reference identity for arrays and objects |
| Partial by design | `value.match(regex)` result | Truthiness is exact; comparison and result chaining are rejected because the evaluator does not expose JavaScript match arrays |
| Partial by design | Regex `u` flag | Accepted for RE2-compatible patterns; JavaScript-only code-point escapes are rejected |
| Unsupported | Arbitrary JavaScript and methods | Assignments, statements, functions, unknown methods, constructors, prototype access, and reflection are rejected |
| Unsupported | JavaScript-only regex features | Lookaround, backreferences, `d`, `v`, and `y` flags are rejected |

RFC functions remain available in Spectral mode. For example,
`match(@.name, 'a.*')` is RFC 9535 full-string matching, whereas
`@.name.match(/a/)` is a Spectral partial search. Spectral `match` returns an
internal boolean-compatible result; comparing it, reading its length, or
chaining from it is rejected rather than approximated.

### Context value types

Spectral mode follows JavaScript object and array property behavior:

- `@property` and `@parentProperty` are strings for mapping entries. YAML keys
are stringified, so an unquoted response key such as `404` works with
`@property.match(/^(4|5)/)`.
- They are numbers for sequence-index comparisons. `@property === 3` can match
the fourth element, while `@property === '3'` cannot. Nimma nevertheless
stringifies a sequence property for its whitelisted `match`, `indexOf`, and
`includes` operations; this engine-specific behavior is pinned in the
differential corpus.
- A method or synthetic property used on the wrong value type evaluates as a
non-match and never panics.

### Regex dialect and safety

Regex flags `i`, `m`, and `s` map to RE2 modes. `g` is accepted because it does
not change boolean match results. `u` is accepted only when the pattern does
not require JavaScript-only Unicode syntax such as `\u{...}`. Flags `d`, `v`,
and `y`, duplicate or unknown flags, lookaround, and backreferences are
rejected during compilation with line and column information.

Spectral mode does not execute JavaScript and does not use reflection. It
rejects arbitrary methods, assignments, statements, constructors,
`__proto__`, prototype traversal, computed constructor access, getters, and
setters before document traversal. Only the operations listed above are
available.

### Compatibility baseline

The checked corpus in
[`pkg/jsonpath/testdata/spectral`](pkg/jsonpath/testdata/spectral) records the
pinned Spectral version, source selector, exact normalized result paths,
multiplicity, and whether Spectral selected Nimma or its JSONPath Plus
fallback. It includes every selector collected from the pinned official
OpenAPI and AsyncAPI rulesets plus focused migration, public-ruleset, Unicode,
security, and Vacuum issue fixtures. A separate public inventory classifies 51
deduplicated selectors from four pinned rulesets. Public Spectral aliases are
expanded before classification, and every source records its repository,
commit, path, and license identifier.

Regenerate the differential results and official-selector inventory with:

```bash
cd pkg/jsonpath/testdata/spectral/generate
npm ci
npm run collect-official
npm run collect-public
npm run generate
```

The currently supported boundary is documented above. Arbitrary JavaScript,
user-provided functions, prototype access, and unsupported regex constructs
are intentionally unsupported.

---

## Overlay Support

This library includes support for YAML overlays, which allow you to apply structured modifications to YAML documents using JSONPath expressions.
Expand Down
36 changes: 36 additions & 0 deletions pkg/jsonpath/config/config.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
package config

import "errors"

// Option configures JSONPath compilation and evaluation.
type Option func(*config)

// WithPropertyNameExtension enables the use of the "~" character to access a property key.
Expand Down Expand Up @@ -28,6 +31,20 @@ func WithStrictRFC9535() Option {
}
}

// WithSpectralCompatibility enables the safe Spectral expression dialect.
//
// Spectral compatibility includes JSONPath Plus context variables, parent
// selection and the property-name extension. It cannot be combined with
// WithStrictRFC9535; callers should validate Config before use. NewPath does
// this automatically.
func WithSpectralCompatibility() Option {
return func(cfg *config) {
cfg.spectralCompatibility = true
cfg.propertyNameExtension = true
}
}

// Config exposes the resolved JSONPath dialect and context-tracking settings.
type Config interface {
PropertyNameEnabled() bool
JSONPathPlusEnabled() bool
Expand All @@ -38,6 +55,7 @@ type config struct {
propertyNameExtension bool
strictRFC9535 bool
lazyContextTracking bool
spectralCompatibility bool
}

func (c *config) PropertyNameEnabled() bool {
Expand All @@ -57,6 +75,24 @@ func (c *config) LazyContextTrackingEnabled() bool {
return c.lazyContextTracking
}

// SpectralCompatibilityEnabled reports whether cfg enables the safe Spectral dialect.
// It is a function rather than a Config method so adding the dialect does not
// break external implementations of the existing Config interface.
func SpectralCompatibilityEnabled(cfg Config) bool {
resolved, ok := cfg.(*config)
return ok && resolved.spectralCompatibility
}

// Validate rejects incompatible dialect options without making option order significant.
func Validate(cfg Config) error {
resolved, ok := cfg.(*config)
if ok && resolved.strictRFC9535 && resolved.spectralCompatibility {
return errors.New("config.WithStrictRFC9535 and config.WithSpectralCompatibility cannot be combined")
}
return nil
}

// New resolves options into an immutable-by-interface configuration view.
func New(opts ...Option) Config {
cfg := &config{}
for _, opt := range opts {
Expand Down
53 changes: 53 additions & 0 deletions pkg/jsonpath/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@ package config

import "testing"

type legacyConfigImplementation struct{}

func (legacyConfigImplementation) PropertyNameEnabled() bool { return false }
func (legacyConfigImplementation) JSONPathPlusEnabled() bool { return true }
func (legacyConfigImplementation) LazyContextTrackingEnabled() bool { return false }

var _ Config = legacyConfigImplementation{}

func TestLazyContextTrackingOption(t *testing.T) {
cfg := New()
if cfg.LazyContextTrackingEnabled() {
Expand All @@ -13,3 +21,48 @@ func TestLazyContextTrackingOption(t *testing.T) {
t.Fatalf("expected lazy context tracking enabled with option")
}
}

func TestSpectralCompatibilityOption(t *testing.T) {
cfg := New(WithSpectralCompatibility())
if !SpectralCompatibilityEnabled(cfg) {
t.Fatal("expected Spectral compatibility to be enabled")
}
if !cfg.JSONPathPlusEnabled() || !cfg.PropertyNameEnabled() {
t.Fatal("expected Spectral compatibility to imply JSONPath Plus and property-name extensions")
}
if err := Validate(cfg); err != nil {
t.Fatalf("unexpected validation error: %v", err)
}
}

func TestSpectralAndExplicitPropertyNameExtensionAreCompatible(t *testing.T) {
cfg := New(WithSpectralCompatibility(), WithPropertyNameExtension())
if err := Validate(cfg); err != nil {
t.Fatalf("redundant property-name option should be valid: %v", err)
}
if !cfg.PropertyNameEnabled() || !SpectralCompatibilityEnabled(cfg) {
t.Fatal("expected both implied settings to remain enabled")
}
}

func TestSpectralAndStrictConflictIsOrderIndependent(t *testing.T) {
tests := []Config{
New(WithSpectralCompatibility(), WithStrictRFC9535()),
New(WithStrictRFC9535(), WithSpectralCompatibility()),
}
for _, cfg := range tests {
if err := Validate(cfg); err == nil {
t.Fatal("expected incompatible dialects to be rejected")
}
}
}

func TestLegacyConfigImplementationsRemainCompatible(t *testing.T) {
cfg := Config(legacyConfigImplementation{})
if SpectralCompatibilityEnabled(cfg) {
t.Fatal("legacy Config implementation unexpectedly enabled Spectral compatibility")
}
if err := Validate(cfg); err != nil {
t.Fatalf("legacy Config implementation failed validation: %v", err)
}
}
21 changes: 18 additions & 3 deletions pkg/jsonpath/context_usage.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ func (q *jsonPathAST) hasPropertyNameReferencesPtr() bool {

// hasPropertyNameReferences reports whether the segment references the property name selector
func (s *segment) hasPropertyNameReferences() bool {
if s.kind == segmentKindProperyName {
if s.kind == segmentKindProperyName || s.kind == segmentKindRecursivePropertyName {
return true
}
if s.child != nil && s.child.hasPropertyNameReferences() {
Expand All @@ -87,14 +87,17 @@ func (s *innerSegment) hasPropertyNameReferences() bool {

// hasPropertyNameReferences reports whether the selector references the property name selector
func (s *selector) hasPropertyNameReferences() bool {
if s.filter != nil && s.filter.hasPropertyNameReferences() {
if s.filter.present() && s.filter.hasPropertyNameReferences() {
return true
}
return false
}

// hasPropertyNameReferences reports whether the filter selector references the property name selector
func (f *filterSelector) hasPropertyNameReferences() bool {
if f.spectralExpression != nil {
return f.spectralExpression.usesPropertyNameSelector
}
if f.expression == nil {
return false
}
Expand Down Expand Up @@ -256,13 +259,25 @@ func (s *innerSegment) collectContextVarUsage(usage *contextVarUsage) {

// collectContextVarUsage records usage from a selector
func (s *selector) collectContextVarUsage(usage *contextVarUsage) {
if s.filter != nil {
if s.filter.present() {
s.filter.collectContextVarUsage(usage)
}
}

// collectContextVarUsage records usage from a filter selector
func (f *filterSelector) collectContextVarUsage(usage *contextVarUsage) {
if f.spectralExpression != nil {
usage.property = usage.property || f.spectralExpression.usage.property
// Parent-property typing needs the container of the parent so sequence
// indexes remain numeric under lazy context tracking.
usage.parent = usage.parent || f.spectralExpression.usage.parent || f.spectralExpression.usage.parentProperty
usage.parentProperty = usage.parentProperty || f.spectralExpression.usage.parentProperty
usage.path = usage.path || f.spectralExpression.usage.path
// Spectral @property is numeric for sequence elements, so lazy mode
// needs the index even when @index is not referenced explicitly.
usage.index = usage.index || f.spectralExpression.usage.index || f.spectralExpression.usage.property
return
}
if f.expression != nil {
f.expression.collectContextVarUsage(usage)
}
Expand Down
46 changes: 23 additions & 23 deletions pkg/jsonpath/context_usage_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ func TestContextUsageAndPropertyReferenceDetection(t *testing.T) {
}

// positive branches across each helper type
truthySelector := &selector{filter: &filterSelector{expression: &logicalOrExpr{
truthySelector := &selector{filter: filterSelector{expression: &logicalOrExpr{
expressions: []*logicalAndExpr{
{
expressions: []*basicExpr{
Expand Down Expand Up @@ -182,7 +182,7 @@ func TestCollectContextVarUsageCoversBranches(t *testing.T) {
},
}
childSelector := &selector{
filter: &filterSelector{expression: &logicalOrExpr{
filter: filterSelector{expression: &logicalOrExpr{
expressions: []*logicalAndExpr{{expressions: []*basicExpr{firstBasic}}},
}},
}
Expand Down Expand Up @@ -221,7 +221,7 @@ func TestCollectContextVarUsageCoversBranches(t *testing.T) {
},
}
descSelector := &selector{
filter: &filterSelector{expression: &logicalOrExpr{
filter: filterSelector{expression: &logicalOrExpr{
expressions: []*logicalAndExpr{{expressions: []*basicExpr{secondBasic}}},
}},
}
Expand Down Expand Up @@ -283,26 +283,26 @@ type bareFilterContext struct {
_index
}

func (b *bareFilterContext) PropertyName() string { return "" }
func (b *bareFilterContext) SetPropertyName(string) {}
func (b *bareFilterContext) Parent() *yaml.Node { return nil }
func (b *bareFilterContext) SetParent(*yaml.Node) {}
func (b *bareFilterContext) ParentPropertyName() string { return "" }
func (b *bareFilterContext) SetParentPropertyName(string) {}
func (b *bareFilterContext) Path() string { return "$" }
func (b *bareFilterContext) PushPathSegment(string) {}
func (b *bareFilterContext) PopPathSegment() {}
func (b *bareFilterContext) SetPendingPathSegment(*yaml.Node, string) {}
func (b *bareFilterContext) GetAndClearPendingPathSegment(*yaml.Node) string { return "" }
func (b *bareFilterContext) SetPendingPropertyName(*yaml.Node, string) {}
func (b *bareFilterContext) GetAndClearPendingPropertyName(*yaml.Node) string { return "" }
func (b *bareFilterContext) Root() *yaml.Node { return nil }
func (b *bareFilterContext) SetRoot(*yaml.Node) {}
func (b *bareFilterContext) Index() int { return -1 }
func (b *bareFilterContext) SetIndex(int) {}
func (b *bareFilterContext) EnableParentTracking() {}
func (b *bareFilterContext) ParentTrackingEnabled() bool { return false }
func (b *bareFilterContext) Clone() FilterContext { return b }
func (b *bareFilterContext) PropertyName() string { return "" }
func (b *bareFilterContext) SetPropertyName(string) {}
func (b *bareFilterContext) Parent() *yaml.Node { return nil }
func (b *bareFilterContext) SetParent(*yaml.Node) {}
func (b *bareFilterContext) ParentPropertyName() string { return "" }
func (b *bareFilterContext) SetParentPropertyName(string) {}
func (b *bareFilterContext) Path() string { return "$" }
func (b *bareFilterContext) PushPathSegment(string) {}
func (b *bareFilterContext) PopPathSegment() {}
func (b *bareFilterContext) SetPendingPathSegment(*yaml.Node, string) {}
func (b *bareFilterContext) GetAndClearPendingPathSegment(*yaml.Node) string { return "" }
func (b *bareFilterContext) SetPendingPropertyName(*yaml.Node, string) {}
func (b *bareFilterContext) GetAndClearPendingPropertyName(*yaml.Node) string { return "" }
func (b *bareFilterContext) Root() *yaml.Node { return nil }
func (b *bareFilterContext) SetRoot(*yaml.Node) {}
func (b *bareFilterContext) Index() int { return -1 }
func (b *bareFilterContext) SetIndex(int) {}
func (b *bareFilterContext) EnableParentTracking() {}
func (b *bareFilterContext) ParentTrackingEnabled() bool { return false }
func (b *bareFilterContext) Clone() FilterContext { return b }

func TestEnableTrackingHelpersNoOpForMissingOptionalMethods(t *testing.T) {
ctx := &bareFilterContext{_index: _index{
Expand Down
Loading
Loading