Regular expressions
===================

The procedures in this section provide access to POSIX regular expression matching.  The regular expression syntax and semantics are far too complex to be described here.

Note: Because the C interface uses ASCII NUL bytes to mark the ends of strings, patterns & strings that contain NUL characters will not work correctly.

Direct POSIX regular expression interface
-----------------------------------------

The first interface to regular expressions is a thin layer over the interface that POSIX provides.  It is exported by the structures posix-regexps & posix.

 -- procedure: make-regexp string option ... returns regexp
 -- procedure: regexp? object returns boolean
     Make-regexp creates a regular expression with the given string pattern.  The arguments after STRING specify various options for the regular expression; see regexp-option below.  The regular expression is not compiled until it is matched against a string, so any errors in the pattern string will not be reported until that point. Regexp? is the disjoint type predicate for regular expression objects.

 -- syntax: regexp-option name returns regexp-option
     Evaluates to a regular expression option, suitable to be passed to make-regexp, with the given name.  The possible option names are:

     extended
          use the extended patterns

     ignore-case
          ignore case differences when matching

     submatches
          report submatches

     newline
          treat newlines specially

 -- procedure: regexp-match regexp string start submatches? starts-line? ends-line? returns boolean or list of matches
     Regexp-match matches REGEXP against the characters in STRING, starting at position START.  If the string does not match the regular expression, regexp-match returns #f. If the string does match, then a list of match records is returned if SUBMATCHES? is true or #t if SUBMATCHES? is false. The first match record gives the location of the substring that matched REGEXP.  If the pattern in REGEXP contained submatches, then the submatches are returned in order, with match records in the positions where submatches succeeded and #f in the positions where submatches failed.

     STARTS-LINE? should be true if STRING starts at the beginning of a line, and ENDS-LINE? should be true if it ends one.

 -- procedure: match? object returns boolean
 -- procedure: match-start match returns integer
 -- procedure: match-end match returns integer
 -- procedure: match-submatches match returns alist
     Match? is the disjoint type predicate for match records.  Match records contain three values: the beginning & end of the substring that matched the pattern and an association list of submatch keys and corresponding match records for any named submatches that also matched. Match-start returns the index of the first character in the matching substring, and match-end gives the index of the first character after the matching substring.  Match-submatches returns the alist of submatches.

High-level regular expression construction
------------------------------------------

This section describes a functional interface for building regular expressions and matching them against strings, higher-level than the direct POSIX interface.  The matching is done using the POSIX regular expression package.  Regular expressions constructed by procedures listed here are compatible with those in the previous section; that is, they satisfy the predicate regexp? from the posix-regexps structure.  These names are exported by the structure regexps.

Character sets
..............

Character sets may be defined using a list of characters and strings, using a range or ranges of characters, or by using set operations on existing character sets.

 -- procedure: set char-or-string ... returns char-set-regexp
 -- procedure: range low-char high-char returns char-set-regexp
 -- procedure: ranges low-char high-char ... returns char-set-regexp
 -- procedure: ascii-range low-char high-char returns char-set-regexp
 -- procedure: ascii-ranges low-char high-char ... returns char-set-regexp
     Set returns a character set that contains all of the character arguments and all of the characters in all of the string arguments. Range returns a character set that contains all characters between LOW-CHAR and HIGH-CHAR, inclusive.  Ranges returns a set that contains all of the characters in the given set of ranges.  Range & ranges use the ordering imposed by char->integer.  Ascii-range & ascii-ranges are like range & ranges, but they use the ASCII ordering. Ranges & ascii-ranges must be given an even number of arguments.  It is an error for a HIGH-CHAR to be less than the preceding LOW-CHAR in the appropriate ordering.

 -- procedure: negate char-set returns char-set-regexp
 -- procedure: union char-seta char-setb returns char-set-regexp
 -- procedure: intersection char-seta char-setb returns char-set-regexp
 -- procedure: subtract char-seta char-setb returns char-set-regexp
     Set operations on character sets.  Negate returns a character set of all characters that are not in CHAR-SET.  Union returns a character set that contains all of the characters in CHAR-SETA and all of the characters in CHAR-SETB.  Intersection returns a character set of all of the characters that are in both CHAR-SETA and CHAR-SETB.  Subtract returns a character set of all the characters in CHAR-SETA that are not also in CHAR-SETB.

 -- character set: lower-case = (set "abcdefghijklmnopqrstuvwxyz")
 -- character set: lower-case = (set "abcdefghijklmnopqrstuvwxyz")
 -- character set: upper-case = (set "ABCDEFGHIJKLMNOPQRSTUVWXYZ")
 -- character set: alphabetic = (union lower-case upper-case)
 -- character set: numeric = (set "0123456789")
 -- character set: alphanumeric = (union alphabetic numeric)
 -- character set: punctuation = (set "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~")
 -- character set: graphic = (union alphanumeric punctuation)
 -- character set: printing = (union graphic (set #\space))
 -- character set: control = (negate printing)
 -- character set: blank = (set #\space (ascii->char 9))     ; ASCII 9 = TAB
 -- character set: whitespace = (union (set #\space) (ascii-range 9 13))
 -- character set: hexdigit = (set "0123456789ABCDEF")
     Predefined character sets.

Anchoring
.........

 -- procedure: string-start returns regexp
 -- procedure: string-end returns regexp
     String-start returns a regular expression that matches the beginning of the string being matched against; string-end returns one that matches the end.

Composite expressions
.....................

 -- procedure: sequence regexp ... returns regexp
 -- procedure: one-of regexp ... returns regexp
     Sequence returns a regular expression that matches concatenation of all of its arguments; one-of returns a regular expression that matches any one of its arguments.

 -- procedure: text string returns regexp
     Returns a regular expression that matches exactly the characters in STRING, in order.

 -- procedure: repeat regexp returns regexp
 -- procedure: repeat count regexp returns regexp
 -- procedure: repeat min max regexp returns regexp
     Repeat returns a regular expression that matches zero or more occurrences of its REGEXP argument.  With only one argument, the result will match REGEXP any number of times.  With two arguments, ie one COUNT argument, the returned regular expression will match REGEXP exactly that number of times.  The final case will match from MIN to MAX repetitions, inclusive.  MAX may be #f, in which case there is no maximum number of matches.  COUNT & MIN must be exact, non-negative integers; MAX should be either #f or an exact, non-negative integer.

Case sensitivity
................

Regular expressions are normally case-sensitive, but case sensitivity can be manipulated simply.

 -- procedure: ignore-case regexp returns regexp
 -- procedure: use-case regexp returns regexp
     The regular expression returned by ignore-case is identical to its argument except that the case will be ignored when matching.  The value returned by use-case is protected from future applications of ignore-case.  The expressions returned by use-case and ignore-case are unaffected by any enclosing uses of these procedures.

     By way of example, the following matches "ab", but not "aB", "Ab", or "AB":

          (text "ab") lisp

          while

               (ignore-case (text "ab")) lisp

               matches all of those, and

                    (ignore-case (sequence (text "a")
                                           (use-case (text "b")))) lisp

                    matches "ab" or "Ab", but not "aB" or "AB".

Submatches and matching
.......................

A subexpression within a larger expression can be marked as a submatch. When an expression is matched against a string, the success or failure of each submatch within that expression is reported, as well as the location of the substring matched by each successful submatch.

 -- procedure: submatch key regexp returns regexp
 -- procedure: no-submatches regexp returns regexp
     Submatch returns a regular expression that is equivalent to REGEXP in every way except that the regular expression returned by submatch will produce a submatch record in the output for the part of the string matched by REGEXP.  No-submatches returns a regular expression that is equivalent to REGEXP in every respect except that all submatches generated by REGEXP will be ignored & removed from the output.

 -- procedure: any-match? regexp string returns boolean
 -- procedure: exact-match? regexp string returns boolean
 -- procedure: match regexp string returns match or #f
     Any-match? returns #t if STRING matches REGEXP or contains a substring that does, or #f if otherwise.  Exact-match? returns #t if STRING matches REGEXP exactly, or #f if it does not.

     Match returns #f if STRING does not match REGEXP, or a match record if it does, as described in the previous section.  Matching occurs according to POSIX.  The match returned is the one with the lowest starting index in STRING.  If there is more than one such match, the longest is returned.  Within that match, the longest possible submatches are returned.

     All three matching procedures cache a compiled version of REGEXP. Subsequent calls with the same input regular expression will be more efficient.

Here are some examples of the high-level regular expression interface:

     (define pattern (text "abc"))

     (any-match? pattern "abc")            => #t
     (any-match? pattern "abx")            => #f
     (any-match? pattern "xxabcxx")        => #t

     (exact-match? pattern "abc")          => #t
     (exact-match? pattern "abx")          => #f
     (exact-match? pattern "xxabcxx")      => #f

     (let ((m (match (sequence (text "ab")
                               (submatch 'foo (text "cd"))
                               (text "ef")))
              "xxabcdefxx"))
       (list m (match-submatches m)))
         => (#{Match 3 9} ((foo . #{Match 5 7})))

     (match-submatches
      (match (sequence (set "a")
                       (one-of (submatch 'foo (text "bc"))
                               (submatch 'bar (text "BC"))))
             "xxxaBCd"))
         => ((bar . #{Match 4 6})) lisp