diff --git a/README.md b/README.md index 22e589c..ce0f4bc 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,42 @@ relative to a base the same way. It also has some functions to work with the It assumes either Windows or POSIX-style separators. +### Glob matching + +`matches?` checks a path against a glob pattern, and `matching` keeps the paths +in an array that match one, in order. Both are lexical as well and never touch +the filesystem, and both take the path first, like everything else here — note +that this is the opposite of `Pattern.matches?` in core. + +```clojure +(Path.matches? "src/main.carp" "src/*.carp") ; => true +(Path.matching &[@"a.carp" @"b.c"] "*.carp") ; => [@"a.carp"] +(Path.matching &[@"a.carp" @"t/b.carp"] "**/*.carp") ; => [@"a.carp" @"t/b.carp"] +``` + +| Form | Meaning | +|----------|------------------------------------------------------| +| `?` | exactly one character, never a separator | +| `*` | zero or more characters, never a separator | +| `**` | zero or more whole segments, as an entire segment | +| `[abc]` | one of the characters in the class | +| `[a-z]` | one character from the range | +| `[!abc]` | one character not in the class (`[^abc]` also works) | +| `\*` | a literal `*`, on POSIX | + +`**` is only a segment wildcard when it is the whole segment, so `**/*.carp` +matches both `a.carp` and `x/y/a.carp`, while `a**b` is just `a*b`. On POSIX a +`\` escapes the next pattern character; on Windows `\` is a separator, so +escaping is disabled there. A trailing `\` and an unterminated `[` are matched +as literal characters, which is why `matches?` is a plain `Bool` and not a +`Result`. Separators are structural and cannot be escaped away: `a\/b` splits +into segments just like `a/b`, and a class holding one, such as `[a/]`, matches +its other members but never the separator. + +Leading dots are not special: `*` matches `.hidden`. Matching happens on the +path exactly as given, with no normalization, so run it through `normalize` +first if `.`, `..` or repeated separators should not get in the way. + Look at [the documentation](https://carpentry.dev/path) for more information.
diff --git a/docs/Path.html b/docs/Path.html index e016ca0..0618bb0 100644 --- a/docs/Path.html +++ b/docs/Path.html @@ -308,6 +308,77 @@

+
+ +

+ matches? +

+
+
+ defn +
+

+ (Fn [(Ref String a), (Ref String b)] Bool) +

+
+                        (matches? p pattern)
+                    
+

+

checks whether the path p matches the glob pattern.

+

The path comes first, like everywhere else in this module — note that this is +the opposite of Pattern.matches? in core:

+
(matches? "src/main.carp" "src/*.carp") ; => true
+
+

The pattern language is:

+
?        exactly one character, never a separator
+*        zero or more characters, never a separator
+**       zero or more whole segments, as an entire segment
+[abc]    one of the characters in the class
+[a-z]    one character from the range
+[!abc]   one character not in the class ([^abc] works too)
+\*       a literal *, on POSIX (see below)
+
+

** is only a segment wildcard when it is the whole segment, so **/*.carp +matches both a.carp and x/y/a.carp, while a**b is just a*b. Inside a +class, a ] directly after the [ (or after the !/^) is a literal ], and +a - first or last is a literal -.

+

On POSIX a \ escapes the next pattern character. On Windows \ is a +separator, so escaping is disabled there. A trailing \ and an unterminated +[ are matched as literal characters, which is why this is a total function and +not a Result.

+

Separators are structural and cannot be escaped away: a\/b splits into +segments just like a/b, and a class holding one, such as [a/], matches its +other members but never the separator.

+

Leading dots are not special: * matches .hidden. Matching happens on the +path exactly as given, with no normalization and no collapsing of repeated +separators — run it through normalize first if you want that.

+ +

+
+
+ +

+ matching +

+
+
+ defn +
+

+ (Fn [(Ref (Array String) a), (Ref String b)] (Array String)) +

+
+                        (matching ps pattern)
+                    
+

+

keeps the paths in ps that match the glob pattern, +preserving their order.

+

It is matches? over an array, which is usually what you want:

+
(matching &[@"a.carp" @"a.c"] "*.carp") ; => [@"a.carp"]
+
+ +

+

diff --git a/path.carp b/path.carp index 1af9466..c8c8423 100644 --- a/path.carp +++ b/path.carp @@ -25,20 +25,24 @@ variable we use on this OS.") (hidden extension-pat) (private sep-string) (hidden sep-string) + (private escapes?) + (hidden escapes?) (windows-only (defn absolute? [p] (Pattern.matches? #"[A-Za-z]:\\" p)) (def separator \\) (def separators [\/ \\]) (def search-path-separator \;) (def extension-pat #"\.[^\\/\.]*$") - (def sep-string "\\")) + (def sep-string "\\") + (def escapes? false)) (posix-only (defn absolute? [p] (String.starts-with? p "/")) (def separator \/) (def separators [\/]) (def search-path-separator \:) (def extension-pat #"\.[^/\.]*$") - (def sep-string "/")) + (def sep-string "/") + (def escapes? true)) (doc relative? "checks whether a path is relative. @@ -225,6 +229,216 @@ drives. Equal paths yield `Just \".\"`.") joined (join &comps)] (Maybe.Just (if (String.empty? &joined) @"." joined)))))))) + (defn- class-end [pcs i] + (let-do [n (Array.length pcs) + j (inc i) + res -1] + (when (and (< j n) + (let [c @(Array.unsafe-nth pcs j)] (or (= c \!) (= c \^)))) + (set! j (inc j))) + (when (and (< j n) (= @(Array.unsafe-nth pcs j) \])) (set! j (inc j))) + (while-do (and (= res -1) (< j n)) + (let [c @(Array.unsafe-nth pcs j)] + (cond + (= c \]) (set! res (inc j)) + (and escapes? (= c \\) (< (inc j) n)) (set! j (+ j 2)) + (set! j (inc j))))) + res)) + + (defn- class-matches? [pcs i end c] + (let-do [j (inc i) + last (dec end) + neg false + found false] + (when-do (and (< j last) + (let [m @(Array.unsafe-nth pcs j)] (or (= m \!) (= m \^)))) + (set! neg true) + (set! j (inc j))) + (while-do (< j last) + (let-do [lo @(Array.unsafe-nth pcs j)] + (set! j (inc j)) + (when-do (and escapes? (= lo \\) (< j last)) + (set! lo @(Array.unsafe-nth pcs j)) + (set! j (inc j))) + (if (and (< (inc j) last) (= @(Array.unsafe-nth pcs j) \-)) + (let-do [hi @(Array.unsafe-nth pcs (inc j))] + (set! j (+ j 2)) + (when-do (and escapes? (= hi \\) (< j last)) + (set! hi @(Array.unsafe-nth pcs j)) + (set! j (inc j))) + (when (and (<= lo c) (<= c hi)) (set! found true))) + (when (= lo c) (set! found true))))) + (if neg (not found) found))) + + ; the first `[` whose class never closes; every later one is unterminated too + (defn- dead-class [pcs] + (let-do [n (Array.length pcs) + i 0 + res -1] + (while-do (and (= res -1) (< i n)) + (let [c @(Array.unsafe-nth pcs i)] + (cond + (and escapes? (= c \\) (< (inc i) n)) (set! i (+ i 2)) + (= c \[) + (let [e (class-end pcs i)] (if (= e -1) (set! res i) (set! i e))) + (set! i (inc i))))) + res)) + + (defn- dead? [dead i] (and (<= 0 dead) (>= i dead))) + + ; index just past the item at `i` when it matches `c`, -1 when it does not + (defn- item-match-end [pcs i dead c] + (let [n (Array.length pcs) + pc @(Array.unsafe-nth pcs i)] + (cond + (and escapes? (= pc \\) (< (inc i) n)) + (if (= @(Array.unsafe-nth pcs (inc i)) c) (+ i 2) -1) + (= pc \?) (inc i) + (and (= pc \[) (not (dead? dead i))) + (let [e (class-end pcs i)] + (cond + (= e -1) (if (= pc c) (inc i) -1) + (class-matches? pcs i e c) e + -1)) + (if (= pc c) (inc i) -1)))) + + (defn- star? [pcs i] (= @(Array.unsafe-nth pcs i) \*)) + + (defn- segment-matches? [scs pcs dead] + (let-do [pn (Array.length pcs) + sn (Array.length scs) + p 0 + s 0 + star -1 + mark 0 + ok true] + (while-do (and ok (< s sn)) + (let [end (if (< p pn) + (item-match-end pcs p dead @(Array.unsafe-nth scs s)) + -1)] + (cond + (and (< p pn) (star? pcs p)) + (do (set! star p) (set! mark s) (set! p (inc p))) + (<= 0 end) (do (set! p end) (set! s (inc s))) + (<= 0 star) + (do (set! mark (inc mark)) (set! s mark) (set! p (inc star))) + (set! ok false)))) + (while-do (and ok (< p pn) (star? pcs p)) (set! p (inc p))) + (and ok (= p pn)))) + + (defn- segment [cs pattern?] + (let-do [n (Array.length cs) + out [] + cur [] + i 0 + classes? pattern?] + (while-do (< i n) + (let-do [c @(Array.unsafe-nth cs i) + esc (and pattern? escapes? (= c \\) (< (inc i) n)) + nxt (if esc @(Array.unsafe-nth cs (inc i)) c) + cend (if (and classes? (= c \[)) (class-end cs i) -1)] + ; once a `[` is unterminated every later one is too + (when (and (= c \[) (= cend -1)) (set! classes? false)) + (cond + (and esc (not (separator? &nxt))) + (do + (set! cur (Array.push-back cur c)) + (set! cur (Array.push-back cur nxt)) + (set! i (+ i 2))) + (/= cend -1) + (while-do (< i cend) + (set! cur (Array.push-back cur @(Array.unsafe-nth cs i))) + (set! i (inc i))) + (separator? &nxt) + (do + (set! out (Array.push-back out @&cur)) + (set! cur []) + (set! i (if esc (+ i 2) (inc i)))) + (do (set! cur (Array.push-back cur c)) (set! i (inc i)))))) + (Array.push-back out cur))) + + (defn- globstar? [seg] + (and (= 2 (Array.length seg)) (star? seg 0) (star? seg 1))) + + (defn- path-matches? [ss ps] + (let-do [pn (Array.length ps) + sn (Array.length ss) + deads (Array.copy-map &dead-class ps) + p 0 + s 0 + star -1 + mark 0 + ok true] + (while-do (and ok (< s sn)) + (cond + (and (< p pn) (globstar? (Array.unsafe-nth ps p))) + (do (set! star p) (set! mark s) (set! p (inc p))) + (and (< p pn) + (segment-matches? (Array.unsafe-nth ss s) + (Array.unsafe-nth ps p) + @(Array.unsafe-nth &deads p))) + (do (set! p (inc p)) (set! s (inc s))) + (<= 0 star) + (do (set! mark (inc mark)) (set! s mark) (set! p (inc star))) + (set! ok false))) + (while-do (and ok (< p pn) (globstar? (Array.unsafe-nth ps p))) + (set! p (inc p))) + (and ok (= p pn)))) + + (doc matches? "checks whether the path `p` matches the glob `pattern`. + +The path comes first, like everywhere else in this module — note that this is +the opposite of `Pattern.matches?` in core: +``` +(matches? \"src/main.carp\" \"src/*.carp\") ; => true +``` + +The pattern language is: +``` +? exactly one character, never a separator +* zero or more characters, never a separator +** zero or more whole segments, as an entire segment +[abc] one of the characters in the class +[a-z] one character from the range +[!abc] one character not in the class ([^abc] works too) +\\* a literal *, on POSIX (see below) +``` + +`**` is only a segment wildcard when it is the whole segment, so `**/*.carp` +matches both `a.carp` and `x/y/a.carp`, while `a**b` is just `a*b`. Inside a +class, a `]` directly after the `[` (or after the `!`/`^`) is a literal `]`, and +a `-` first or last is a literal `-`. + +On POSIX a `\\` escapes the next pattern character. On Windows `\\` is a +separator, so escaping is disabled there. A trailing `\\` and an unterminated +`[` are matched as literal characters, which is why this is a total function and +not a `Result`. + +Separators are structural and cannot be escaped away: `a\\/b` splits into +segments just like `a/b`, and a class holding one, such as `[a/]`, matches its +other members but never the separator. + +Leading dots are not special: `*` matches `.hidden`. Matching happens on the +path exactly as given, with no normalization and no collapsing of repeated +separators — run it through [normalize](#normalize) first if you want that.") + (defn matches? [p pattern] + (path-matches? &(segment &(String.chars p) false) + &(segment &(String.chars pattern) true))) + + (doc matching "keeps the paths in `ps` that match the glob `pattern`, +preserving their order. + +It is [matches?](#matches?) over an array, which is usually what you want: +``` +(matching &[@\"a.carp\" @\"a.c\"] \"*.carp\") ; => [@\"a.carp\"] +```") + (defn matching [ps pattern] + (let-do [out []] + (for [i 0 (Array.length ps)] + (let [p (Array.unsafe-nth ps i)] + (when (matches? p pattern) (set! out (Array.push-back out @p))))) + out)) + (doc split-search-path "splits a `PATH` environment variable `p`.") (defn split-search-path [p] (String.split-by p &[search-path-separator])) (doc get-search-path "gets the `PATH` environment variable and splits it.") diff --git a/test/path.carp b/test/path.carp index 508ed77..e809bd7 100644 --- a/test/path.carp +++ b/test/path.carp @@ -182,7 +182,182 @@ (assert-equal test &[@"/usr/bin" @"/usr/local/bin"] &(split-search-path "/usr/bin:/usr/local/bin") - "split-search-path works")) + "split-search-path works") + (assert-true test (matches? "a.carp" "a.carp") "matches? matches a literal") + (assert-false test + (matches? "a.carp" "a.car") + "matches? rejects a literal mismatch") + (assert-true test + (matches? "a.carp" "?.carp") + "matches? matches ? against one character") + (assert-false test + (matches? "ab.carp" "?.carp") + "matches? does not match ? against two characters") + (assert-false test + (matches? "" "?") + "matches? does not match ? against nothing") + (assert-true test + (matches? "abc.carp" "*.carp") + "matches? matches * against many characters") + (assert-true test (matches? "" "*") "matches? matches * against nothing") + (assert-true test + (matches? "src/main.carp" "src/*.carp") + "matches? matches segment by segment") + (assert-false test + (matches? "a/b" "a*b") + "matches? does not let * cross a separator") + (assert-false test + (matches? "a/b" "a?b") + "matches? does not let ? cross a separator") + (assert-false test + (matches? "a/b/c" "*") + "matches? does not match * against a nested path") + (assert-true test + (matches? "a.carp" "**/*.carp") + "matches? matches a leading ** against zero segments") + (assert-true test + (matches? "x/y/a.carp" "**/*.carp") + "matches? matches a leading ** against many segments") + (assert-true test + (matches? "a/b" "a/**/b") + "matches? matches an inner ** against zero segments") + (assert-true test + (matches? "a/x/y/b" "a/**/b") + "matches? matches an inner ** against many segments") + (assert-true test + (matches? "a" "a/**") + "matches? matches a trailing ** against zero segments") + (assert-true test (matches? "a/b/c" "a/**") "matches? matches a trailing **") + (assert-false test + (matches? "a/b" "x/**") + "matches? rejects a ** pattern with the wrong prefix") + (assert-true test + (matches? "axxb" "a**b") + "matches? treats a partial ** as a plain *") + (assert-false test + (matches? "a/b" "a**b") + "matches? does not let a partial ** cross a separator") + (assert-true test (matches? "abc" "[abc]bc") "matches? matches a class") + (assert-false test + (matches? "dbc" "[abc]bc") + "matches? rejects a character outside a class") + (assert-true test (matches? "m" "[a-z]") "matches? matches a range") + (assert-false test + (matches? "M" "[a-z]") + "matches? rejects a character outside a range") + (assert-true test (matches? "d" "[!abc]") "matches? negates a class with !") + (assert-false test + (matches? "a" "[!abc]") + "matches? rejects a member of a negated class") + (assert-true test (matches? "d" "[^abc]") "matches? negates a class with ^") + (assert-false test (matches? "b" "[^a-c]") "matches? negates a range") + (assert-true test + (matches? "]" "[]]") + "matches? reads a leading ] in a class as a literal") + (assert-true test + (matches? "a" "[]a]") + "matches? keeps reading a class after a literal ]") + (assert-false test + (matches? "]" "[!]a]") + "matches? reads a ] after a negation as a class member") + (assert-true test + (matches? "b" "[!]a]") + "matches? terminates a negated class at the second ]") + (assert-true test + (matches? "-" "[a-]") + "matches? reads a trailing - in a class as a literal") + (assert-false test + (matches? "b" "[a-]") + "matches? does not read a trailing - as a range") + (assert-true test + (matches? "-" "[-a]") + "matches? reads a leading - in a class as a literal") + (assert-true test (matches? "*" "\\*") "matches? escapes a * into a literal") + (assert-false test + (matches? "a" "\\*") + "matches? does not let an escaped * match another character") + (assert-true test + (matches? "[abc]" "\\[abc]") + "matches? escapes a [ into a literal") + (assert-true test + (matches? "[abc" "[abc") + "matches? reads an unterminated [ as a literal") + (assert-false test + (matches? "a" "[abc") + "matches? does not read an unterminated [ as a class") + (assert-true test + (matches? "[a[b" "[a[b") + "matches? reads a [ after an unterminated [ as a literal too") + (assert-true test + (matches? "a/b" "a\\/b") + "matches? splits on an escaped separator like a plain one") + (assert-true test + (matches? "/a" "\\/a") + "matches? splits on a leading escaped separator") + (assert-false test + (matches? "a/b/c" "a\\/b") + "matches? does not let an escaped separator match a whole path") + (assert-true test + (matches? "a" "[a/]") + "matches? does not split a class that holds a separator") + (assert-true test + (matches? "abc" "[!/]*") + "matches? does not split a negated class that holds a separator") + (assert-false test + (matches? "/" "[a/]") + "matches? never matches a separator with a class") + (assert-false test + (matches? "a/b" "a[/]b") + "matches? keeps a class in one segment") + (assert-true test + (matches? "[a/b" "[a/b") + "matches? splits on a separator in an unterminated class") + (assert-true test + (matches? "c/b" "[!a]/b") + "matches? splits on a separator after a class") + (assert-true test (matches? "" "") "matches? matches two empty strings") + (assert-false test + (matches? "a" "") + "matches? rejects an empty pattern against a non-empty path") + (assert-false test + (matches? "" "a") + "matches? rejects a non-empty pattern against an empty path") + (assert-true test + (matches? ".hidden" "*") + "matches? does not treat a leading dot as special") + (assert-true test + (matches? "ü" "?") + "matches? matches ? against a multi-byte character") + (assert-false test + (matches? "grüße.txt" "gr??ße.*") + "matches? counts characters, not bytes") + (assert-true test + (matches? "ü" "[üö]") + "matches? matches a class against a multi-byte character") + (assert-false test + (matches? "a" "[üö]") + "matches? rejects a character outside a multi-byte class") + (assert-false test + (matches? "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" "*a*a*a*a*a*a*b") + "matches? backtracks linearly") + (assert-equal test + &[@"a.carp" @"d.carp"] + &(matching &[@"a.carp" @"b.c" @"d.carp"] "*.carp") + "matching keeps the matching paths in order") + (assert-equal test + &[@"test/a.carp" @"test/b/c.carp"] + &(matching + &[@"test/a.carp" @"src/a.carp" @"test/b/c.carp"] + "test/**/*.carp") + "matching filters a nested listing") + (assert-equal test + &(the (Array String) []) + &(matching &[@"a.carp"] "*.rs") + "matching returns an empty array if nothing matches") + (assert-equal test + &(the (Array String) []) + &(matching &(the (Array String) []) "*") + "matching handles an empty array")) ()) (windows-only (deftest test