1 <html> 2 <head> 3 <title>pcre2pattern specification</title> 4 </head> 5 <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> 6 <h1>pcre2pattern man page</h1> 7 <p> 8 Return to the <a href="index.html">PCRE2 index page</a>. 9 </p> 10 <p> 11 This page is part of the PCRE2 HTML documentation. It was generated 12 automatically from the original man page. If there is any nonsense in it, 13 please consult the man page, in case the conversion went wrong. 14 <br> 15 <ul> 16 <li><a name="TOC1" href="#SEC1">PCRE2 REGULAR EXPRESSION DETAILS</a> 17 <li><a name="TOC2" href="#SEC2">SPECIAL START-OF-PATTERN ITEMS</a> 18 <li><a name="TOC3" href="#SEC3">EBCDIC CHARACTER CODES</a> 19 <li><a name="TOC4" href="#SEC4">CHARACTERS AND METACHARACTERS</a> 20 <li><a name="TOC5" href="#SEC5">BACKSLASH</a> 21 <li><a name="TOC6" href="#SEC6">CIRCUMFLEX AND DOLLAR</a> 22 <li><a name="TOC7" href="#SEC7">FULL STOP (PERIOD, DOT) AND \N</a> 23 <li><a name="TOC8" href="#SEC8">MATCHING A SINGLE CODE UNIT</a> 24 <li><a name="TOC9" href="#SEC9">SQUARE BRACKETS AND CHARACTER CLASSES</a> 25 <li><a name="TOC10" href="#SEC10">POSIX CHARACTER CLASSES</a> 26 <li><a name="TOC11" href="#SEC11">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a> 27 <li><a name="TOC12" href="#SEC12">VERTICAL BAR</a> 28 <li><a name="TOC13" href="#SEC13">INTERNAL OPTION SETTING</a> 29 <li><a name="TOC14" href="#SEC14">SUBPATTERNS</a> 30 <li><a name="TOC15" href="#SEC15">DUPLICATE SUBPATTERN NUMBERS</a> 31 <li><a name="TOC16" href="#SEC16">NAMED SUBPATTERNS</a> 32 <li><a name="TOC17" href="#SEC17">REPETITION</a> 33 <li><a name="TOC18" href="#SEC18">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a> 34 <li><a name="TOC19" href="#SEC19">BACK REFERENCES</a> 35 <li><a name="TOC20" href="#SEC20">ASSERTIONS</a> 36 <li><a name="TOC21" href="#SEC21">CONDITIONAL SUBPATTERNS</a> 37 <li><a name="TOC22" href="#SEC22">COMMENTS</a> 38 <li><a name="TOC23" href="#SEC23">RECURSIVE PATTERNS</a> 39 <li><a name="TOC24" href="#SEC24">SUBPATTERNS AS SUBROUTINES</a> 40 <li><a name="TOC25" href="#SEC25">ONIGURUMA SUBROUTINE SYNTAX</a> 41 <li><a name="TOC26" href="#SEC26">CALLOUTS</a> 42 <li><a name="TOC27" href="#SEC27">BACKTRACKING CONTROL</a> 43 <li><a name="TOC28" href="#SEC28">SEE ALSO</a> 44 <li><a name="TOC29" href="#SEC29">AUTHOR</a> 45 <li><a name="TOC30" href="#SEC30">REVISION</a> 46 </ul> 47 <br><a name="SEC1" href="#TOC1">PCRE2 REGULAR EXPRESSION DETAILS</a><br> 48 <P> 49 The syntax and semantics of the regular expressions that are supported by PCRE2 50 are described in detail below. There is a quick-reference syntax summary in the 51 <a href="pcre2syntax.html"><b>pcre2syntax</b></a> 52 page. PCRE2 tries to match Perl syntax and semantics as closely as it can. 53 PCRE2 also supports some alternative regular expression syntax (which does not 54 conflict with the Perl syntax) in order to provide some compatibility with 55 regular expressions in Python, .NET, and Oniguruma. 56 </P> 57 <P> 58 Perl's regular expressions are described in its own documentation, and regular 59 expressions in general are covered in a number of books, some of which have 60 copious examples. Jeffrey Friedl's "Mastering Regular Expressions", published 61 by O'Reilly, covers regular expressions in great detail. This description of 62 PCRE2's regular expressions is intended as reference material. 63 </P> 64 <P> 65 This document discusses the patterns that are supported by PCRE2 when its main 66 matching function, <b>pcre2_match()</b>, is used. PCRE2 also has an alternative 67 matching function, <b>pcre2_dfa_match()</b>, which matches using a different 68 algorithm that is not Perl-compatible. Some of the features discussed below are 69 not available when DFA matching is used. The advantages and disadvantages of 70 the alternative function, and how it differs from the normal function, are 71 discussed in the 72 <a href="pcre2matching.html"><b>pcre2matching</b></a> 73 page. 74 </P> 75 <br><a name="SEC2" href="#TOC1">SPECIAL START-OF-PATTERN ITEMS</a><br> 76 <P> 77 A number of options that can be passed to <b>pcre2_compile()</b> can also be set 78 by special items at the start of a pattern. These are not Perl-compatible, but 79 are provided to make these options accessible to pattern writers who are not 80 able to change the program that processes the pattern. Any number of these 81 items may appear, but they must all be together right at the start of the 82 pattern string, and the letters must be in upper case. 83 </P> 84 <br><b> 85 UTF support 86 </b><br> 87 <P> 88 In the 8-bit and 16-bit PCRE2 libraries, characters may be coded either as 89 single code units, or as multiple UTF-8 or UTF-16 code units. UTF-32 can be 90 specified for the 32-bit library, in which case it constrains the character 91 values to valid Unicode code points. To process UTF strings, PCRE2 must be 92 built to include Unicode support (which is the default). When using UTF strings 93 you must either call the compiling function with the PCRE2_UTF option, or the 94 pattern must start with the special sequence (*UTF), which is equivalent to 95 setting the relevant option. How setting a UTF mode affects pattern matching is 96 mentioned in several places below. There is also a summary of features in the 97 <a href="pcre2unicode.html"><b>pcre2unicode</b></a> 98 page. 99 </P> 100 <P> 101 Some applications that allow their users to supply patterns may wish to 102 restrict them to non-UTF data for security reasons. If the PCRE2_NEVER_UTF 103 option is passed to <b>pcre2_compile()</b>, (*UTF) is not allowed, and its 104 appearance in a pattern causes an error. 105 </P> 106 <br><b> 107 Unicode property support 108 </b><br> 109 <P> 110 Another special sequence that may appear at the start of a pattern is (*UCP). 111 This has the same effect as setting the PCRE2_UCP option: it causes sequences 112 such as \d and \w to use Unicode properties to determine character types, 113 instead of recognizing only characters with codes less than 256 via a lookup 114 table. 115 </P> 116 <P> 117 Some applications that allow their users to supply patterns may wish to 118 restrict them for security reasons. If the PCRE2_NEVER_UCP option is passed to 119 <b>pcre2_compile()</b>, (*UCP) is not allowed, and its appearance in a pattern 120 causes an error. 121 </P> 122 <br><b> 123 Locking out empty string matching 124 </b><br> 125 <P> 126 Starting a pattern with (*NOTEMPTY) or (*NOTEMPTY_ATSTART) has the same effect 127 as passing the PCRE2_NOTEMPTY or PCRE2_NOTEMPTY_ATSTART option to whichever 128 matching function is subsequently called to match the pattern. These options 129 lock out the matching of empty strings, either entirely, or only at the start 130 of the subject. 131 </P> 132 <br><b> 133 Disabling auto-possessification 134 </b><br> 135 <P> 136 If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as setting 137 the PCRE2_NO_AUTO_POSSESS option. This stops PCRE2 from making quantifiers 138 possessive when what follows cannot match the repeated item. For example, by 139 default a+b is treated as a++b. For more details, see the 140 <a href="pcre2api.html"><b>pcre2api</b></a> 141 documentation. 142 </P> 143 <br><b> 144 Disabling start-up optimizations 145 </b><br> 146 <P> 147 If a pattern starts with (*NO_START_OPT), it has the same effect as setting the 148 PCRE2_NO_START_OPTIMIZE option. This disables several optimizations for quickly 149 reaching "no match" results. For more details, see the 150 <a href="pcre2api.html"><b>pcre2api</b></a> 151 documentation. 152 </P> 153 <br><b> 154 Disabling automatic anchoring 155 </b><br> 156 <P> 157 If a pattern starts with (*NO_DOTSTAR_ANCHOR), it has the same effect as 158 setting the PCRE2_NO_DOTSTAR_ANCHOR option. This disables optimizations that 159 apply to patterns whose top-level branches all start with .* (match any number 160 of arbitrary characters). For more details, see the 161 <a href="pcre2api.html"><b>pcre2api</b></a> 162 documentation. 163 </P> 164 <br><b> 165 Disabling JIT compilation 166 </b><br> 167 <P> 168 If a pattern that starts with (*NO_JIT) is successfully compiled, an attempt by 169 the application to apply the JIT optimization by calling 170 <b>pcre2_jit_compile()</b> is ignored. 171 </P> 172 <br><b> 173 Setting match and recursion limits 174 </b><br> 175 <P> 176 The caller of <b>pcre2_match()</b> can set a limit on the number of times the 177 internal <b>match()</b> function is called and on the maximum depth of 178 recursive calls. These facilities are provided to catch runaway matches that 179 are provoked by patterns with huge matching trees (a typical example is a 180 pattern with nested unlimited repeats) and to avoid running out of system stack 181 by too much recursion. When one of these limits is reached, <b>pcre2_match()</b> 182 gives an error return. The limits can also be set by items at the start of the 183 pattern of the form 184 <pre> 185 (*LIMIT_MATCH=d) 186 (*LIMIT_RECURSION=d) 187 </pre> 188 where d is any number of decimal digits. However, the value of the setting must 189 be less than the value set (or defaulted) by the caller of <b>pcre2_match()</b> 190 for it to have any effect. In other words, the pattern writer can lower the 191 limits set by the programmer, but not raise them. If there is more than one 192 setting of one of these limits, the lower value is used. 193 <a name="newlines"></a></P> 194 <br><b> 195 Newline conventions 196 </b><br> 197 <P> 198 PCRE2 supports five different conventions for indicating line breaks in 199 strings: a single CR (carriage return) character, a single LF (linefeed) 200 character, the two-character sequence CRLF, any of the three preceding, or any 201 Unicode newline sequence. The 202 <a href="pcre2api.html"><b>pcre2api</b></a> 203 page has 204 <a href="pcre2api.html#newlines">further discussion</a> 205 about newlines, and shows how to set the newline convention when calling 206 <b>pcre2_compile()</b>. 207 </P> 208 <P> 209 It is also possible to specify a newline convention by starting a pattern 210 string with one of the following five sequences: 211 <pre> 212 (*CR) carriage return 213 (*LF) linefeed 214 (*CRLF) carriage return, followed by linefeed 215 (*ANYCRLF) any of the three above 216 (*ANY) all Unicode newline sequences 217 </pre> 218 These override the default and the options given to the compiling function. For 219 example, on a Unix system where LF is the default newline sequence, the pattern 220 <pre> 221 (*CR)a.b 222 </pre> 223 changes the convention to CR. That pattern matches "a\nb" because LF is no 224 longer a newline. If more than one of these settings is present, the last one 225 is used. 226 </P> 227 <P> 228 The newline convention affects where the circumflex and dollar assertions are 229 true. It also affects the interpretation of the dot metacharacter when 230 PCRE2_DOTALL is not set, and the behaviour of \N. However, it does not affect 231 what the \R escape sequence matches. By default, this is any Unicode newline 232 sequence, for Perl compatibility. However, this can be changed; see the 233 description of \R in the section entitled 234 <a href="#newlineseq">"Newline sequences"</a> 235 below. A change of \R setting can be combined with a change of newline 236 convention. 237 </P> 238 <br><b> 239 Specifying what \R matches 240 </b><br> 241 <P> 242 It is possible to restrict \R to match only CR, LF, or CRLF (instead of the 243 complete set of Unicode line endings) by setting the option PCRE2_BSR_ANYCRLF 244 at compile time. This effect can also be achieved by starting a pattern with 245 (*BSR_ANYCRLF). For completeness, (*BSR_UNICODE) is also recognized, 246 corresponding to PCRE2_BSR_UNICODE. 247 </P> 248 <br><a name="SEC3" href="#TOC1">EBCDIC CHARACTER CODES</a><br> 249 <P> 250 PCRE2 can be compiled to run in an environment that uses EBCDIC as its 251 character code rather than ASCII or Unicode (typically a mainframe system). In 252 the sections below, character code values are ASCII or Unicode; in an EBCDIC 253 environment these characters may have different code values, and there are no 254 code points greater than 255. 255 </P> 256 <br><a name="SEC4" href="#TOC1">CHARACTERS AND METACHARACTERS</a><br> 257 <P> 258 A regular expression is a pattern that is matched against a subject string from 259 left to right. Most characters stand for themselves in a pattern, and match the 260 corresponding characters in the subject. As a trivial example, the pattern 261 <pre> 262 The quick brown fox 263 </pre> 264 matches a portion of a subject string that is identical to itself. When 265 caseless matching is specified (the PCRE2_CASELESS option), letters are matched 266 independently of case. 267 </P> 268 <P> 269 The power of regular expressions comes from the ability to include alternatives 270 and repetitions in the pattern. These are encoded in the pattern by the use of 271 <i>metacharacters</i>, which do not stand for themselves but instead are 272 interpreted in some special way. 273 </P> 274 <P> 275 There are two different sets of metacharacters: those that are recognized 276 anywhere in the pattern except within square brackets, and those that are 277 recognized within square brackets. Outside square brackets, the metacharacters 278 are as follows: 279 <pre> 280 \ general escape character with several uses 281 ^ assert start of string (or line, in multiline mode) 282 $ assert end of string (or line, in multiline mode) 283 . match any character except newline (by default) 284 [ start character class definition 285 | start of alternative branch 286 ( start subpattern 287 ) end subpattern 288 ? extends the meaning of ( 289 also 0 or 1 quantifier 290 also quantifier minimizer 291 * 0 or more quantifier 292 + 1 or more quantifier 293 also "possessive quantifier" 294 { start min/max quantifier 295 </pre> 296 Part of a pattern that is in square brackets is called a "character class". In 297 a character class the only metacharacters are: 298 <pre> 299 \ general escape character 300 ^ negate the class, but only if the first character 301 - indicates character range 302 [ POSIX character class (only if followed by POSIX syntax) 303 ] terminates the character class 304 </pre> 305 The following sections describe the use of each of the metacharacters. 306 </P> 307 <br><a name="SEC5" href="#TOC1">BACKSLASH</a><br> 308 <P> 309 The backslash character has several uses. Firstly, if it is followed by a 310 character that is not a number or a letter, it takes away any special meaning 311 that character may have. This use of backslash as an escape character applies 312 both inside and outside character classes. 313 </P> 314 <P> 315 For example, if you want to match a * character, you write \* in the pattern. 316 This escaping action applies whether or not the following character would 317 otherwise be interpreted as a metacharacter, so it is always safe to precede a 318 non-alphanumeric with backslash to specify that it stands for itself. In 319 particular, if you want to match a backslash, you write \\. 320 </P> 321 <P> 322 In a UTF mode, only ASCII numbers and letters have any special meaning after a 323 backslash. All other characters (in particular, those whose codepoints are 324 greater than 127) are treated as literals. 325 </P> 326 <P> 327 If a pattern is compiled with the PCRE2_EXTENDED option, most white space in 328 the pattern (other than in a character class), and characters between a # 329 outside a character class and the next newline, inclusive, are ignored. An 330 escaping backslash can be used to include a white space or # character as part 331 of the pattern. 332 </P> 333 <P> 334 If you want to remove the special meaning from a sequence of characters, you 335 can do so by putting them between \Q and \E. This is different from Perl in 336 that $ and @ are handled as literals in \Q...\E sequences in PCRE2, whereas 337 in Perl, $ and @ cause variable interpolation. Note the following examples: 338 <pre> 339 Pattern PCRE2 matches Perl matches 340 341 \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz 342 \Qabc\$xyz\E abc\$xyz abc\$xyz 343 \Qabc\E\$\Qxyz\E abc$xyz abc$xyz 344 </pre> 345 The \Q...\E sequence is recognized both inside and outside character classes. 346 An isolated \E that is not preceded by \Q is ignored. If \Q is not followed 347 by \E later in the pattern, the literal interpretation continues to the end of 348 the pattern (that is, \E is assumed at the end). If the isolated \Q is inside 349 a character class, this causes an error, because the character class is not 350 terminated. 351 <a name="digitsafterbackslash"></a></P> 352 <br><b> 353 Non-printing characters 354 </b><br> 355 <P> 356 A second use of backslash provides a way of encoding non-printing characters 357 in patterns in a visible manner. There is no restriction on the appearance of 358 non-printing characters in a pattern, but when a pattern is being prepared by 359 text editing, it is often easier to use one of the following escape sequences 360 than the binary character it represents. In an ASCII or Unicode environment, 361 these escapes are as follows: 362 <pre> 363 \a alarm, that is, the BEL character (hex 07) 364 \cx "control-x", where x is any printable ASCII character 365 \e escape (hex 1B) 366 \f form feed (hex 0C) 367 \n linefeed (hex 0A) 368 \r carriage return (hex 0D) 369 \t tab (hex 09) 370 \0dd character with octal code 0dd 371 \ddd character with octal code ddd, or back reference 372 \o{ddd..} character with octal code ddd.. 373 \xhh character with hex code hh 374 \x{hhh..} character with hex code hhh.. (default mode) 375 \uhhhh character with hex code hhhh (when PCRE2_ALT_BSUX is set) 376 </pre> 377 The precise effect of \cx on ASCII characters is as follows: if x is a lower 378 case letter, it is converted to upper case. Then bit 6 of the character (hex 379 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), 380 but \c{ becomes hex 3B ({ is 7B), and \c; becomes hex 7B (; is 3B). If the 381 code unit following \c has a value less than 32 or greater than 126, a 382 compile-time error occurs. This locks out non-printable ASCII characters in all 383 modes. 384 </P> 385 <P> 386 When PCRE2 is compiled in EBCDIC mode, \a, \e, \f, \n, \r, and \t 387 generate the appropriate EBCDIC code values. The \c escape is processed 388 as specified for Perl in the <b>perlebcdic</b> document. The only characters 389 that are allowed after \c are A-Z, a-z, or one of @, [, \, ], ^, _, or ?. Any 390 other character provokes a compile-time error. The sequence \@ encodes 391 character code 0; the letters (in either case) encode characters 1-26 (hex 01 392 to hex 1A); [, \, ], ^, and _ encode characters 27-31 (hex 1B to hex 1F), and 393 \? becomes either 255 (hex FF) or 95 (hex 5F). 394 </P> 395 <P> 396 Thus, apart from \?, these escapes generate the same character code values as 397 they do in an ASCII environment, though the meanings of the values mostly 398 differ. For example, \G always generates code value 7, which is BEL in ASCII 399 but DEL in EBCDIC. 400 </P> 401 <P> 402 The sequence \? generates DEL (127, hex 7F) in an ASCII environment, but 403 because 127 is not a control character in EBCDIC, Perl makes it generate the 404 APC character. Unfortunately, there are several variants of EBCDIC. In most of 405 them the APC character has the value 255 (hex FF), but in the one Perl calls 406 POSIX-BC its value is 95 (hex 5F). If certain other characters have POSIX-BC 407 values, PCRE2 makes \? generate 95; otherwise it generates 255. 408 </P> 409 <P> 410 After \0 up to two further octal digits are read. If there are fewer than two 411 digits, just those that are present are used. Thus the sequence \0\x\015 412 specifies two binary zeros followed by a CR character (code value 13). Make 413 sure you supply two digits after the initial zero if the pattern character that 414 follows is itself an octal digit. 415 </P> 416 <P> 417 The escape \o must be followed by a sequence of octal digits, enclosed in 418 braces. An error occurs if this is not the case. This escape is a recent 419 addition to Perl; it provides way of specifying character code points as octal 420 numbers greater than 0777, and it also allows octal numbers and back references 421 to be unambiguously specified. 422 </P> 423 <P> 424 For greater clarity and unambiguity, it is best to avoid following \ by a 425 digit greater than zero. Instead, use \o{} or \x{} to specify character 426 numbers, and \g{} to specify back references. The following paragraphs 427 describe the old, ambiguous syntax. 428 </P> 429 <P> 430 The handling of a backslash followed by a digit other than 0 is complicated, 431 and Perl has changed over time, causing PCRE2 also to change. 432 </P> 433 <P> 434 Outside a character class, PCRE2 reads the digit and any following digits as a 435 decimal number. If the number is less than 10, begins with the digit 8 or 9, or 436 if there are at least that many previous capturing left parentheses in the 437 expression, the entire sequence is taken as a <i>back reference</i>. A 438 description of how this works is given 439 <a href="#backreferences">later,</a> 440 following the discussion of 441 <a href="#subpattern">parenthesized subpatterns.</a> 442 Otherwise, up to three octal digits are read to form a character code. 443 </P> 444 <P> 445 Inside a character class, PCRE2 handles \8 and \9 as the literal characters 446 "8" and "9", and otherwise reads up to three octal digits following the 447 backslash, using them to generate a data character. Any subsequent digits stand 448 for themselves. For example, outside a character class: 449 <pre> 450 \040 is another way of writing an ASCII space 451 \40 is the same, provided there are fewer than 40 previous capturing subpatterns 452 \7 is always a back reference 453 \11 might be a back reference, or another way of writing a tab 454 \011 is always a tab 455 \0113 is a tab followed by the character "3" 456 \113 might be a back reference, otherwise the character with octal code 113 457 \377 might be a back reference, otherwise the value 255 (decimal) 458 \81 is always a back reference .sp 459 </pre> 460 Note that octal values of 100 or greater that are specified using this syntax 461 must not be introduced by a leading zero, because no more than three octal 462 digits are ever read. 463 </P> 464 <P> 465 By default, after \x that is not followed by {, from zero to two hexadecimal 466 digits are read (letters can be in upper or lower case). Any number of 467 hexadecimal digits may appear between \x{ and }. If a character other than 468 a hexadecimal digit appears between \x{ and }, or if there is no terminating 469 }, an error occurs. 470 </P> 471 <P> 472 If the PCRE2_ALT_BSUX option is set, the interpretation of \x is as just 473 described only when it is followed by two hexadecimal digits. Otherwise, it 474 matches a literal "x" character. In this mode mode, support for code points 475 greater than 256 is provided by \u, which must be followed by four hexadecimal 476 digits; otherwise it matches a literal "u" character. 477 </P> 478 <P> 479 Characters whose value is less than 256 can be defined by either of the two 480 syntaxes for \x (or by \u in PCRE2_ALT_BSUX mode). There is no difference in 481 the way they are handled. For example, \xdc is exactly the same as \x{dc} (or 482 \u00dc in PCRE2_ALT_BSUX mode). 483 </P> 484 <br><b> 485 Constraints on character values 486 </b><br> 487 <P> 488 Characters that are specified using octal or hexadecimal numbers are 489 limited to certain values, as follows: 490 <pre> 491 8-bit non-UTF mode less than 0x100 492 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint 493 16-bit non-UTF mode less than 0x10000 494 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint 495 32-bit non-UTF mode less than 0x100000000 496 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint 497 </pre> 498 Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-called 499 "surrogate" codepoints), and 0xffef. 500 </P> 501 <br><b> 502 Escape sequences in character classes 503 </b><br> 504 <P> 505 All the sequences that define a single character value can be used both inside 506 and outside character classes. In addition, inside a character class, \b is 507 interpreted as the backspace character (hex 08). 508 </P> 509 <P> 510 \N is not allowed in a character class. \B, \R, and \X are not special 511 inside a character class. Like other unrecognized alphabetic escape sequences, 512 they cause an error. Outside a character class, these sequences have different 513 meanings. 514 </P> 515 <br><b> 516 Unsupported escape sequences 517 </b><br> 518 <P> 519 In Perl, the sequences \l, \L, \u, and \U are recognized by its string 520 handler and used to modify the case of following characters. By default, PCRE2 521 does not support these escape sequences. However, if the PCRE2_ALT_BSUX option 522 is set, \U matches a "U" character, and \u can be used to define a character 523 by code point, as described in the previous section. 524 </P> 525 <br><b> 526 Absolute and relative back references 527 </b><br> 528 <P> 529 The sequence \g followed by an unsigned or a negative number, optionally 530 enclosed in braces, is an absolute or relative back reference. A named back 531 reference can be coded as \g{name}. Back references are discussed 532 <a href="#backreferences">later,</a> 533 following the discussion of 534 <a href="#subpattern">parenthesized subpatterns.</a> 535 </P> 536 <br><b> 537 Absolute and relative subroutine calls 538 </b><br> 539 <P> 540 For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or 541 a number enclosed either in angle brackets or single quotes, is an alternative 542 syntax for referencing a subpattern as a "subroutine". Details are discussed 543 <a href="#onigurumasubroutines">later.</a> 544 Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> 545 synonymous. The former is a back reference; the latter is a 546 <a href="#subpatternsassubroutines">subroutine</a> 547 call. 548 <a name="genericchartypes"></a></P> 549 <br><b> 550 Generic character types 551 </b><br> 552 <P> 553 Another use of backslash is for specifying generic character types: 554 <pre> 555 \d any decimal digit 556 \D any character that is not a decimal digit 557 \h any horizontal white space character 558 \H any character that is not a horizontal white space character 559 \s any white space character 560 \S any character that is not a white space character 561 \v any vertical white space character 562 \V any character that is not a vertical white space character 563 \w any "word" character 564 \W any "non-word" character 565 </pre> 566 There is also the single sequence \N, which matches a non-newline character. 567 This is the same as 568 <a href="#fullstopdot">the "." metacharacter</a> 569 when PCRE2_DOTALL is not set. Perl also uses \N to match characters by name; 570 PCRE2 does not support this. 571 </P> 572 <P> 573 Each pair of lower and upper case escape sequences partitions the complete set 574 of characters into two disjoint sets. Any given character matches one, and only 575 one, of each pair. The sequences can appear both inside and outside character 576 classes. They each match one character of the appropriate type. If the current 577 matching point is at the end of the subject string, all of them fail, because 578 there is no character to match. 579 </P> 580 <P> 581 The default \s characters are HT (9), LF (10), VT (11), FF (12), CR (13), and 582 space (32), which are defined as white space in the "C" locale. This list may 583 vary if locale-specific matching is taking place. For example, in some locales 584 the "non-breaking space" character (\xA0) is recognized as white space, and in 585 others the VT character is not. 586 </P> 587 <P> 588 A "word" character is an underscore or any character that is a letter or digit. 589 By default, the definition of letters and digits is controlled by PCRE2's 590 low-valued character tables, and may vary if locale-specific matching is taking 591 place (see 592 <a href="pcre2api.html#localesupport">"Locale support"</a> 593 in the 594 <a href="pcre2api.html"><b>pcre2api</b></a> 595 page). For example, in a French locale such as "fr_FR" in Unix-like systems, 596 or "french" in Windows, some character codes greater than 127 are used for 597 accented letters, and these are then matched by \w. The use of locales with 598 Unicode is discouraged. 599 </P> 600 <P> 601 By default, characters whose code points are greater than 127 never match \d, 602 \s, or \w, and always match \D, \S, and \W, although this may be different 603 for characters in the range 128-255 when locale-specific matching is happening. 604 These escape sequences retain their original meanings from before Unicode 605 support was available, mainly for efficiency reasons. If the PCRE2_UCP option 606 is set, the behaviour is changed so that Unicode properties are used to 607 determine character types, as follows: 608 <pre> 609 \d any character that matches \p{Nd} (decimal digit) 610 \s any character that matches \p{Z} or \h or \v 611 \w any character that matches \p{L} or \p{N}, plus underscore 612 </pre> 613 The upper case escapes match the inverse sets of characters. Note that \d 614 matches only decimal digits, whereas \w matches any Unicode digit, as well as 615 any Unicode letter, and underscore. Note also that PCRE2_UCP affects \b, and 616 \B because they are defined in terms of \w and \W. Matching these sequences 617 is noticeably slower when PCRE2_UCP is set. 618 </P> 619 <P> 620 The sequences \h, \H, \v, and \V, in contrast to the other sequences, which 621 match only ASCII characters by default, always match a specific list of code 622 points, whether or not PCRE2_UCP is set. The horizontal space characters are: 623 <pre> 624 U+0009 Horizontal tab (HT) 625 U+0020 Space 626 U+00A0 Non-break space 627 U+1680 Ogham space mark 628 U+180E Mongolian vowel separator 629 U+2000 En quad 630 U+2001 Em quad 631 U+2002 En space 632 U+2003 Em space 633 U+2004 Three-per-em space 634 U+2005 Four-per-em space 635 U+2006 Six-per-em space 636 U+2007 Figure space 637 U+2008 Punctuation space 638 U+2009 Thin space 639 U+200A Hair space 640 U+202F Narrow no-break space 641 U+205F Medium mathematical space 642 U+3000 Ideographic space 643 </pre> 644 The vertical space characters are: 645 <pre> 646 U+000A Linefeed (LF) 647 U+000B Vertical tab (VT) 648 U+000C Form feed (FF) 649 U+000D Carriage return (CR) 650 U+0085 Next line (NEL) 651 U+2028 Line separator 652 U+2029 Paragraph separator 653 </pre> 654 In 8-bit, non-UTF-8 mode, only the characters with code points less than 256 655 are relevant. 656 <a name="newlineseq"></a></P> 657 <br><b> 658 Newline sequences 659 </b><br> 660 <P> 661 Outside a character class, by default, the escape sequence \R matches any 662 Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent to the 663 following: 664 <pre> 665 (?>\r\n|\n|\x0b|\f|\r|\x85) 666 </pre> 667 This is an example of an "atomic group", details of which are given 668 <a href="#atomicgroup">below.</a> 669 This particular group matches either the two-character sequence CR followed by 670 LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, 671 U+000B), FF (form feed, U+000C), CR (carriage return, U+000D), or NEL (next 672 line, U+0085). Because this is an atomic group, the two-character sequence is 673 treated as a single unit that cannot be split. 674 </P> 675 <P> 676 In other modes, two additional characters whose codepoints are greater than 255 677 are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). 678 Unicode support is not needed for these characters to be recognized. 679 </P> 680 <P> 681 It is possible to restrict \R to match only CR, LF, or CRLF (instead of the 682 complete set of Unicode line endings) by setting the option PCRE2_BSR_ANYCRLF 683 at compile time. (BSR is an abbrevation for "backslash R".) This can be made 684 the default when PCRE2 is built; if this is the case, the other behaviour can 685 be requested via the PCRE2_BSR_UNICODE option. It is also possible to specify 686 these settings by starting a pattern string with one of the following 687 sequences: 688 <pre> 689 (*BSR_ANYCRLF) CR, LF, or CRLF only 690 (*BSR_UNICODE) any Unicode newline sequence 691 </pre> 692 These override the default and the options given to the compiling function. 693 Note that these special settings, which are not Perl-compatible, are recognized 694 only at the very start of a pattern, and that they must be in upper case. If 695 more than one of them is present, the last one is used. They can be combined 696 with a change of newline convention; for example, a pattern can start with: 697 <pre> 698 (*ANY)(*BSR_ANYCRLF) 699 </pre> 700 They can also be combined with the (*UTF) or (*UCP) special sequences. Inside a 701 character class, \R is treated as an unrecognized escape sequence, and causes 702 an error. 703 <a name="uniextseq"></a></P> 704 <br><b> 705 Unicode character properties 706 </b><br> 707 <P> 708 When PCRE2 is built with Unicode support (the default), three additional escape 709 sequences that match characters with specific properties are available. In 710 8-bit non-UTF-8 mode, these sequences are of course limited to testing 711 characters whose codepoints are less than 256, but they do work in this mode. 712 The extra escape sequences are: 713 <pre> 714 \p{<i>xx</i>} a character with the <i>xx</i> property 715 \P{<i>xx</i>} a character without the <i>xx</i> property 716 \X a Unicode extended grapheme cluster 717 </pre> 718 The property names represented by <i>xx</i> above are limited to the Unicode 719 script names, the general category properties, "Any", which matches any 720 character (including newline), and some special PCRE2 properties (described 721 in the 722 <a href="#extraprops">next section).</a> 723 Other Perl properties such as "InMusicalSymbols" are not supported by PCRE2. 724 Note that \P{Any} does not match any characters, so always causes a match 725 failure. 726 </P> 727 <P> 728 Sets of Unicode characters are defined as belonging to certain scripts. A 729 character from one of these sets can be matched using a script name. For 730 example: 731 <pre> 732 \p{Greek} 733 \P{Han} 734 </pre> 735 Those that are not part of an identified script are lumped together as 736 "Common". The current list of scripts is: 737 </P> 738 <P> 739 Ahom, 740 Anatolian_Hieroglyphs, 741 Arabic, 742 Armenian, 743 Avestan, 744 Balinese, 745 Bamum, 746 Bassa_Vah, 747 Batak, 748 Bengali, 749 Bopomofo, 750 Brahmi, 751 Braille, 752 Buginese, 753 Buhid, 754 Canadian_Aboriginal, 755 Carian, 756 Caucasian_Albanian, 757 Chakma, 758 Cham, 759 Cherokee, 760 Common, 761 Coptic, 762 Cuneiform, 763 Cypriot, 764 Cyrillic, 765 Deseret, 766 Devanagari, 767 Duployan, 768 Egyptian_Hieroglyphs, 769 Elbasan, 770 Ethiopic, 771 Georgian, 772 Glagolitic, 773 Gothic, 774 Grantha, 775 Greek, 776 Gujarati, 777 Gurmukhi, 778 Han, 779 Hangul, 780 Hanunoo, 781 Hatran, 782 Hebrew, 783 Hiragana, 784 Imperial_Aramaic, 785 Inherited, 786 Inscriptional_Pahlavi, 787 Inscriptional_Parthian, 788 Javanese, 789 Kaithi, 790 Kannada, 791 Katakana, 792 Kayah_Li, 793 Kharoshthi, 794 Khmer, 795 Khojki, 796 Khudawadi, 797 Lao, 798 Latin, 799 Lepcha, 800 Limbu, 801 Linear_A, 802 Linear_B, 803 Lisu, 804 Lycian, 805 Lydian, 806 Mahajani, 807 Malayalam, 808 Mandaic, 809 Manichaean, 810 Meetei_Mayek, 811 Mende_Kikakui, 812 Meroitic_Cursive, 813 Meroitic_Hieroglyphs, 814 Miao, 815 Modi, 816 Mongolian, 817 Mro, 818 Multani, 819 Myanmar, 820 Nabataean, 821 New_Tai_Lue, 822 Nko, 823 Ogham, 824 Ol_Chiki, 825 Old_Hungarian, 826 Old_Italic, 827 Old_North_Arabian, 828 Old_Permic, 829 Old_Persian, 830 Old_South_Arabian, 831 Old_Turkic, 832 Oriya, 833 Osmanya, 834 Pahawh_Hmong, 835 Palmyrene, 836 Pau_Cin_Hau, 837 Phags_Pa, 838 Phoenician, 839 Psalter_Pahlavi, 840 Rejang, 841 Runic, 842 Samaritan, 843 Saurashtra, 844 Sharada, 845 Shavian, 846 Siddham, 847 SignWriting, 848 Sinhala, 849 Sora_Sompeng, 850 Sundanese, 851 Syloti_Nagri, 852 Syriac, 853 Tagalog, 854 Tagbanwa, 855 Tai_Le, 856 Tai_Tham, 857 Tai_Viet, 858 Takri, 859 Tamil, 860 Telugu, 861 Thaana, 862 Thai, 863 Tibetan, 864 Tifinagh, 865 Tirhuta, 866 Ugaritic, 867 Vai, 868 Warang_Citi, 869 Yi. 870 </P> 871 <P> 872 Each character has exactly one Unicode general category property, specified by 873 a two-letter abbreviation. For compatibility with Perl, negation can be 874 specified by including a circumflex between the opening brace and the property 875 name. For example, \p{^Lu} is the same as \P{Lu}. 876 </P> 877 <P> 878 If only one letter is specified with \p or \P, it includes all the general 879 category properties that start with that letter. In this case, in the absence 880 of negation, the curly brackets in the escape sequence are optional; these two 881 examples have the same effect: 882 <pre> 883 \p{L} 884 \pL 885 </pre> 886 The following general category property codes are supported: 887 <pre> 888 C Other 889 Cc Control 890 Cf Format 891 Cn Unassigned 892 Co Private use 893 Cs Surrogate 894 895 L Letter 896 Ll Lower case letter 897 Lm Modifier letter 898 Lo Other letter 899 Lt Title case letter 900 Lu Upper case letter 901 902 M Mark 903 Mc Spacing mark 904 Me Enclosing mark 905 Mn Non-spacing mark 906 907 N Number 908 Nd Decimal number 909 Nl Letter number 910 No Other number 911 912 P Punctuation 913 Pc Connector punctuation 914 Pd Dash punctuation 915 Pe Close punctuation 916 Pf Final punctuation 917 Pi Initial punctuation 918 Po Other punctuation 919 Ps Open punctuation 920 921 S Symbol 922 Sc Currency symbol 923 Sk Modifier symbol 924 Sm Mathematical symbol 925 So Other symbol 926 927 Z Separator 928 Zl Line separator 929 Zp Paragraph separator 930 Zs Space separator 931 </pre> 932 The special property L& is also supported: it matches a character that has 933 the Lu, Ll, or Lt property, in other words, a letter that is not classified as 934 a modifier or "other". 935 </P> 936 <P> 937 The Cs (Surrogate) property applies only to characters in the range U+D800 to 938 U+DFFF. Such characters are not valid in Unicode strings and so 939 cannot be tested by PCRE2, unless UTF validity checking has been turned off 940 (see the discussion of PCRE2_NO_UTF_CHECK in the 941 <a href="pcre2api.html"><b>pcre2api</b></a> 942 page). Perl does not support the Cs property. 943 </P> 944 <P> 945 The long synonyms for property names that Perl supports (such as \p{Letter}) 946 are not supported by PCRE2, nor is it permitted to prefix any of these 947 properties with "Is". 948 </P> 949 <P> 950 No character that is in the Unicode table has the Cn (unassigned) property. 951 Instead, this property is assumed for any code point that is not in the 952 Unicode table. 953 </P> 954 <P> 955 Specifying caseless matching does not affect these escape sequences. For 956 example, \p{Lu} always matches only upper case letters. This is different from 957 the behaviour of current versions of Perl. 958 </P> 959 <P> 960 Matching characters by Unicode property is not fast, because PCRE2 has to do a 961 multistage table lookup in order to find a character's property. That is why 962 the traditional escape sequences such as \d and \w do not use Unicode 963 properties in PCRE2 by default, though you can make them do so by setting the 964 PCRE2_UCP option or by starting the pattern with (*UCP). 965 </P> 966 <br><b> 967 Extended grapheme clusters 968 </b><br> 969 <P> 970 The \X escape matches any number of Unicode characters that form an "extended 971 grapheme cluster", and treats the sequence as an atomic group 972 <a href="#atomicgroup">(see below).</a> 973 Unicode supports various kinds of composite character by giving each character 974 a grapheme breaking property, and having rules that use these properties to 975 define the boundaries of extended grapheme clusters. \X always matches at 976 least one character. Then it decides whether to add additional characters 977 according to the following rules for ending a cluster: 978 </P> 979 <P> 980 1. End at the end of the subject string. 981 </P> 982 <P> 983 2. Do not end between CR and LF; otherwise end after any control character. 984 </P> 985 <P> 986 3. Do not break Hangul (a Korean script) syllable sequences. Hangul characters 987 are of five types: L, V, T, LV, and LVT. An L character may be followed by an 988 L, V, LV, or LVT character; an LV or V character may be followed by a V or T 989 character; an LVT or T character may be follwed only by a T character. 990 </P> 991 <P> 992 4. Do not end before extending characters or spacing marks. Characters with 993 the "mark" property always have the "extend" grapheme breaking property. 994 </P> 995 <P> 996 5. Do not end after prepend characters. 997 </P> 998 <P> 999 6. Otherwise, end the cluster. 1000 <a name="extraprops"></a></P> 1001 <br><b> 1002 PCRE2's additional properties 1003 </b><br> 1004 <P> 1005 As well as the standard Unicode properties described above, PCRE2 supports four 1006 more that make it possible to convert traditional escape sequences such as \w 1007 and \s to use Unicode properties. PCRE2 uses these non-standard, non-Perl 1008 properties internally when PCRE2_UCP is set. However, they may also be used 1009 explicitly. These properties are: 1010 <pre> 1011 Xan Any alphanumeric character 1012 Xps Any POSIX space character 1013 Xsp Any Perl space character 1014 Xwd Any Perl "word" character 1015 </pre> 1016 Xan matches characters that have either the L (letter) or the N (number) 1017 property. Xps matches the characters tab, linefeed, vertical tab, form feed, or 1018 carriage return, and any other character that has the Z (separator) property. 1019 Xsp is the same as Xps; in PCRE1 it used to exclude vertical tab, for Perl 1020 compatibility, but Perl changed. Xwd matches the same characters as Xan, plus 1021 underscore. 1022 </P> 1023 <P> 1024 There is another non-standard property, Xuc, which matches any character that 1025 can be represented by a Universal Character Name in C++ and other programming 1026 languages. These are the characters $, @, ` (grave accent), and all characters 1027 with Unicode code points greater than or equal to U+00A0, except for the 1028 surrogates U+D800 to U+DFFF. Note that most base (ASCII) characters are 1029 excluded. (Universal Character Names are of the form \uHHHH or \UHHHHHHHH 1030 where H is a hexadecimal digit. Note that the Xuc property does not match these 1031 sequences but the characters that they represent.) 1032 <a name="resetmatchstart"></a></P> 1033 <br><b> 1034 Resetting the match start 1035 </b><br> 1036 <P> 1037 The escape sequence \K causes any previously matched characters not to be 1038 included in the final matched sequence. For example, the pattern: 1039 <pre> 1040 foo\Kbar 1041 </pre> 1042 matches "foobar", but reports that it has matched "bar". This feature is 1043 similar to a lookbehind assertion 1044 <a href="#lookbehind">(described below).</a> 1045 However, in this case, the part of the subject before the real match does not 1046 have to be of fixed length, as lookbehind assertions do. The use of \K does 1047 not interfere with the setting of 1048 <a href="#subpattern">captured substrings.</a> 1049 For example, when the pattern 1050 <pre> 1051 (foo)\Kbar 1052 </pre> 1053 matches "foobar", the first substring is still set to "foo". 1054 </P> 1055 <P> 1056 Perl documents that the use of \K within assertions is "not well defined". In 1057 PCRE2, \K is acted upon when it occurs inside positive assertions, but is 1058 ignored in negative assertions. Note that when a pattern such as (?=ab\K) 1059 matches, the reported start of the match can be greater than the end of the 1060 match. 1061 <a name="smallassertions"></a></P> 1062 <br><b> 1063 Simple assertions 1064 </b><br> 1065 <P> 1066 The final use of backslash is for certain simple assertions. An assertion 1067 specifies a condition that has to be met at a particular point in a match, 1068 without consuming any characters from the subject string. The use of 1069 subpatterns for more complicated assertions is described 1070 <a href="#bigassertions">below.</a> 1071 The backslashed assertions are: 1072 <pre> 1073 \b matches at a word boundary 1074 \B matches when not at a word boundary 1075 \A matches at the start of the subject 1076 \Z matches at the end of the subject 1077 also matches before a newline at the end of the subject 1078 \z matches only at the end of the subject 1079 \G matches at the first matching position in the subject 1080 </pre> 1081 Inside a character class, \b has a different meaning; it matches the backspace 1082 character. If any other of these assertions appears in a character class, an 1083 "invalid escape sequence" error is generated. 1084 </P> 1085 <P> 1086 A word boundary is a position in the subject string where the current character 1087 and the previous character do not both match \w or \W (i.e. one matches 1088 \w and the other matches \W), or the start or end of the string if the 1089 first or last character matches \w, respectively. In a UTF mode, the meanings 1090 of \w and \W can be changed by setting the PCRE2_UCP option. When this is 1091 done, it also affects \b and \B. Neither PCRE2 nor Perl has a separate "start 1092 of word" or "end of word" metasequence. However, whatever follows \b normally 1093 determines which it is. For example, the fragment \ba matches "a" at the start 1094 of a word. 1095 </P> 1096 <P> 1097 The \A, \Z, and \z assertions differ from the traditional circumflex and 1098 dollar (described in the next section) in that they only ever match at the very 1099 start and end of the subject string, whatever options are set. Thus, they are 1100 independent of multiline mode. These three assertions are not affected by the 1101 PCRE2_NOTBOL or PCRE2_NOTEOL options, which affect only the behaviour of the 1102 circumflex and dollar metacharacters. However, if the <i>startoffset</i> 1103 argument of <b>pcre2_match()</b> is non-zero, indicating that matching is to 1104 start at a point other than the beginning of the subject, \A can never match. 1105 The difference between \Z and \z is that \Z matches before a newline at the 1106 end of the string as well as at the very end, whereas \z matches only at the 1107 end. 1108 </P> 1109 <P> 1110 The \G assertion is true only when the current matching position is at the 1111 start point of the match, as specified by the <i>startoffset</i> argument of 1112 <b>pcre2_match()</b>. It differs from \A when the value of <i>startoffset</i> is 1113 non-zero. By calling <b>pcre2_match()</b> multiple times with appropriate 1114 arguments, you can mimic Perl's /g option, and it is in this kind of 1115 implementation where \G can be useful. 1116 </P> 1117 <P> 1118 Note, however, that PCRE2's interpretation of \G, as the start of the current 1119 match, is subtly different from Perl's, which defines it as the end of the 1120 previous match. In Perl, these can be different when the previously matched 1121 string was empty. Because PCRE2 does just one match at a time, it cannot 1122 reproduce this behaviour. 1123 </P> 1124 <P> 1125 If all the alternatives of a pattern begin with \G, the expression is anchored 1126 to the starting match position, and the "anchored" flag is set in the compiled 1127 regular expression. 1128 </P> 1129 <br><a name="SEC6" href="#TOC1">CIRCUMFLEX AND DOLLAR</a><br> 1130 <P> 1131 The circumflex and dollar metacharacters are zero-width assertions. That is, 1132 they test for a particular condition being true without consuming any 1133 characters from the subject string. These two metacharacters are concerned with 1134 matching the starts and ends of lines. If the newline convention is set so that 1135 only the two-character sequence CRLF is recognized as a newline, isolated CR 1136 and LF characters are treated as ordinary data characters, and are not 1137 recognized as newlines. 1138 </P> 1139 <P> 1140 Outside a character class, in the default matching mode, the circumflex 1141 character is an assertion that is true only if the current matching point is at 1142 the start of the subject string. If the <i>startoffset</i> argument of 1143 <b>pcre2_match()</b> is non-zero, or if PCRE2_NOTBOL is set, circumflex can 1144 never match if the PCRE2_MULTILINE option is unset. Inside a character class, 1145 circumflex has an entirely different meaning 1146 <a href="#characterclass">(see below).</a> 1147 </P> 1148 <P> 1149 Circumflex need not be the first character of the pattern if a number of 1150 alternatives are involved, but it should be the first thing in each alternative 1151 in which it appears if the pattern is ever to match that branch. If all 1152 possible alternatives start with a circumflex, that is, if the pattern is 1153 constrained to match only at the start of the subject, it is said to be an 1154 "anchored" pattern. (There are also other constructs that can cause a pattern 1155 to be anchored.) 1156 </P> 1157 <P> 1158 The dollar character is an assertion that is true only if the current matching 1159 point is at the end of the subject string, or immediately before a newline at 1160 the end of the string (by default), unless PCRE2_NOTEOL is set. Note, however, 1161 that it does not actually match the newline. Dollar need not be the last 1162 character of the pattern if a number of alternatives are involved, but it 1163 should be the last item in any branch in which it appears. Dollar has no 1164 special meaning in a character class. 1165 </P> 1166 <P> 1167 The meaning of dollar can be changed so that it matches only at the very end of 1168 the string, by setting the PCRE2_DOLLAR_ENDONLY option at compile time. This 1169 does not affect the \Z assertion. 1170 </P> 1171 <P> 1172 The meanings of the circumflex and dollar metacharacters are changed if the 1173 PCRE2_MULTILINE option is set. When this is the case, a dollar character 1174 matches before any newlines in the string, as well as at the very end, and a 1175 circumflex matches immediately after internal newlines as well as at the start 1176 of the subject string. It does not match after a newline that ends the string, 1177 for compatibility with Perl. However, this can be changed by setting the 1178 PCRE2_ALT_CIRCUMFLEX option. 1179 </P> 1180 <P> 1181 For example, the pattern /^abc$/ matches the subject string "def\nabc" (where 1182 \n represents a newline) in multiline mode, but not otherwise. Consequently, 1183 patterns that are anchored in single line mode because all branches start with 1184 ^ are not anchored in multiline mode, and a match for circumflex is possible 1185 when the <i>startoffset</i> argument of <b>pcre2_match()</b> is non-zero. The 1186 PCRE2_DOLLAR_ENDONLY option is ignored if PCRE2_MULTILINE is set. 1187 </P> 1188 <P> 1189 When the newline convention (see 1190 <a href="#newlines">"Newline conventions"</a> 1191 below) recognizes the two-character sequence CRLF as a newline, this is 1192 preferred, even if the single characters CR and LF are also recognized as 1193 newlines. For example, if the newline convention is "any", a multiline mode 1194 circumflex matches before "xyz" in the string "abc\r\nxyz" rather than after 1195 CR, even though CR on its own is a valid newline. (It also matches at the very 1196 start of the string, of course.) 1197 </P> 1198 <P> 1199 Note that the sequences \A, \Z, and \z can be used to match the start and 1200 end of the subject in both modes, and if all branches of a pattern start with 1201 \A it is always anchored, whether or not PCRE2_MULTILINE is set. 1202 <a name="fullstopdot"></a></P> 1203 <br><a name="SEC7" href="#TOC1">FULL STOP (PERIOD, DOT) AND \N</a><br> 1204 <P> 1205 Outside a character class, a dot in the pattern matches any one character in 1206 the subject string except (by default) a character that signifies the end of a 1207 line. 1208 </P> 1209 <P> 1210 When a line ending is defined as a single character, dot never matches that 1211 character; when the two-character sequence CRLF is used, dot does not match CR 1212 if it is immediately followed by LF, but otherwise it matches all characters 1213 (including isolated CRs and LFs). When any Unicode line endings are being 1214 recognized, dot does not match CR or LF or any of the other line ending 1215 characters. 1216 </P> 1217 <P> 1218 The behaviour of dot with regard to newlines can be changed. If the 1219 PCRE2_DOTALL option is set, a dot matches any one character, without exception. 1220 If the two-character sequence CRLF is present in the subject string, it takes 1221 two dots to match it. 1222 </P> 1223 <P> 1224 The handling of dot is entirely independent of the handling of circumflex and 1225 dollar, the only relationship being that they both involve newlines. Dot has no 1226 special meaning in a character class. 1227 </P> 1228 <P> 1229 The escape sequence \N behaves like a dot, except that it is not affected by 1230 the PCRE2_DOTALL option. In other words, it matches any character except one 1231 that signifies the end of a line. Perl also uses \N to match characters by 1232 name; PCRE2 does not support this. 1233 </P> 1234 <br><a name="SEC8" href="#TOC1">MATCHING A SINGLE CODE UNIT</a><br> 1235 <P> 1236 Outside a character class, the escape sequence \C matches any one code unit, 1237 whether or not a UTF mode is set. In the 8-bit library, one code unit is one 1238 byte; in the 16-bit library it is a 16-bit unit; in the 32-bit library it is a 1239 32-bit unit. Unlike a dot, \C always matches line-ending characters. The 1240 feature is provided in Perl in order to match individual bytes in UTF-8 mode, 1241 but it is unclear how it can usefully be used. 1242 </P> 1243 <P> 1244 Because \C breaks up characters into individual code units, matching one unit 1245 with \C in UTF-8 or UTF-16 mode means that the rest of the string may start 1246 with a malformed UTF character. This has undefined results, because PCRE2 1247 assumes that it is matching character by character in a valid UTF string (by 1248 default it checks the subject string's validity at the start of processing 1249 unless the PCRE2_NO_UTF_CHECK option is used). 1250 </P> 1251 <P> 1252 An application can lock out the use of \C by setting the 1253 PCRE2_NEVER_BACKSLASH_C option when compiling a pattern. It is also possible to 1254 build PCRE2 with the use of \C permanently disabled. 1255 </P> 1256 <P> 1257 PCRE2 does not allow \C to appear in lookbehind assertions 1258 <a href="#lookbehind">(described below)</a> 1259 in UTF-8 or UTF-16 modes, because this would make it impossible to calculate 1260 the length of the lookbehind. Neither the alternative matching function 1261 <b>pcre2_dfa_match()</b> nor the JIT optimizer support \C in these UTF modes. 1262 The former gives a match-time error; the latter fails to optimize and so the 1263 match is always run using the interpreter. 1264 </P> 1265 <P> 1266 In the 32-bit library, however, \C is always supported (when not explicitly 1267 locked out) because it always matches a single code unit, whether or not UTF-32 1268 is specified. 1269 </P> 1270 <P> 1271 In general, the \C escape sequence is best avoided. However, one way of using 1272 it that avoids the problem of malformed UTF-8 or UTF-16 characters is to use a 1273 lookahead to check the length of the next character, as in this pattern, which 1274 could be used with a UTF-8 string (ignore white space and line breaks): 1275 <pre> 1276 (?| (?=[\x00-\x7f])(\C) | 1277 (?=[\x80-\x{7ff}])(\C)(\C) | 1278 (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) | 1279 (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C)) 1280 </pre> 1281 In this example, a group that starts with (?| resets the capturing parentheses 1282 numbers in each alternative (see 1283 <a href="#dupsubpatternnumber">"Duplicate Subpattern Numbers"</a> 1284 below). The assertions at the start of each branch check the next UTF-8 1285 character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The 1286 character's individual bytes are then captured by the appropriate number of 1287 \C groups. 1288 <a name="characterclass"></a></P> 1289 <br><a name="SEC9" href="#TOC1">SQUARE BRACKETS AND CHARACTER CLASSES</a><br> 1290 <P> 1291 An opening square bracket introduces a character class, terminated by a closing 1292 square bracket. A closing square bracket on its own is not special by default. 1293 If a closing square bracket is required as a member of the class, it should be 1294 the first data character in the class (after an initial circumflex, if present) 1295 or escaped with a backslash. This means that, by default, an empty class cannot 1296 be defined. However, if the PCRE2_ALLOW_EMPTY_CLASS option is set, a closing 1297 square bracket at the start does end the (empty) class. 1298 </P> 1299 <P> 1300 A character class matches a single character in the subject. A matched 1301 character must be in the set of characters defined by the class, unless the 1302 first character in the class definition is a circumflex, in which case the 1303 subject character must not be in the set defined by the class. If a circumflex 1304 is actually required as a member of the class, ensure it is not the first 1305 character, or escape it with a backslash. 1306 </P> 1307 <P> 1308 For example, the character class [aeiou] matches any lower case vowel, while 1309 [^aeiou] matches any character that is not a lower case vowel. Note that a 1310 circumflex is just a convenient notation for specifying the characters that 1311 are in the class by enumerating those that are not. A class that starts with a 1312 circumflex is not an assertion; it still consumes a character from the subject 1313 string, and therefore it fails if the current pointer is at the end of the 1314 string. 1315 </P> 1316 <P> 1317 When caseless matching is set, any letters in a class represent both their 1318 upper case and lower case versions, so for example, a caseless [aeiou] matches 1319 "A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a 1320 caseful version would. 1321 </P> 1322 <P> 1323 Characters that might indicate line breaks are never treated in any special way 1324 when matching character classes, whatever line-ending sequence is in use, and 1325 whatever setting of the PCRE2_DOTALL and PCRE2_MULTILINE options is used. A 1326 class such as [^a] always matches one of these characters. 1327 </P> 1328 <P> 1329 The minus (hyphen) character can be used to specify a range of characters in a 1330 character class. For example, [d-m] matches any letter between d and m, 1331 inclusive. If a minus character is required in a class, it must be escaped with 1332 a backslash or appear in a position where it cannot be interpreted as 1333 indicating a range, typically as the first or last character in the class, or 1334 immediately after a range. For example, [b-d-z] matches letters in the range b 1335 to d, a hyphen character, or z. 1336 </P> 1337 <P> 1338 It is not possible to have the literal character "]" as the end character of a 1339 range. A pattern such as [W-]46] is interpreted as a class of two characters 1340 ("W" and "-") followed by a literal string "46]", so it would match "W46]" or 1341 "-46]". However, if the "]" is escaped with a backslash it is interpreted as 1342 the end of range, so [W-\]46] is interpreted as a class containing a range 1343 followed by two other characters. The octal or hexadecimal representation of 1344 "]" can also be used to end a range. 1345 </P> 1346 <P> 1347 An error is generated if a POSIX character class (see below) or an escape 1348 sequence other than one that defines a single character appears at a point 1349 where a range ending character is expected. For example, [z-\xff] is valid, 1350 but [A-\d] and [A-[:digit:]] are not. 1351 </P> 1352 <P> 1353 Ranges normally include all code points between the start and end characters, 1354 inclusive. They can also be used for code points specified numerically, for 1355 example [\000-\037]. Ranges can include any characters that are valid for the 1356 current mode. 1357 </P> 1358 <P> 1359 There is a special case in EBCDIC environments for ranges whose end points are 1360 both specified as literal letters in the same case. For compatibility with 1361 Perl, EBCDIC code points within the range that are not letters are omitted. For 1362 example, [h-k] matches only four characters, even though the codes for h and k 1363 are 0x88 and 0x92, a range of 11 code points. However, if the range is 1364 specified numerically, for example, [\x88-\x92] or [h-\x92], all code points 1365 are included. 1366 </P> 1367 <P> 1368 If a range that includes letters is used when caseless matching is set, it 1369 matches the letters in either case. For example, [W-c] is equivalent to 1370 [][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if character 1371 tables for a French locale are in use, [\xc8-\xcb] matches accented E 1372 characters in both cases. 1373 </P> 1374 <P> 1375 The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, 1376 \V, \w, and \W may appear in a character class, and add the characters that 1377 they match to the class. For example, [\dABCDEF] matches any hexadecimal 1378 digit. In UTF modes, the PCRE2_UCP option affects the meanings of \d, \s, \w 1379 and their upper case partners, just as it does when they appear outside a 1380 character class, as described in the section entitled 1381 <a href="#genericchartypes">"Generic character types"</a> 1382 above. The escape sequence \b has a different meaning inside a character 1383 class; it matches the backspace character. The sequences \B, \N, \R, and \X 1384 are not special inside a character class. Like any other unrecognized escape 1385 sequences, they cause an error. 1386 </P> 1387 <P> 1388 A circumflex can conveniently be used with the upper case character types to 1389 specify a more restricted set of characters than the matching lower case type. 1390 For example, the class [^\W_] matches any letter or digit, but not underscore, 1391 whereas [\w] includes underscore. A positive character class should be read as 1392 "something OR something OR ..." and a negative class as "NOT something AND NOT 1393 something AND NOT ...". 1394 </P> 1395 <P> 1396 The only metacharacters that are recognized in character classes are backslash, 1397 hyphen (only where it can be interpreted as specifying a range), circumflex 1398 (only at the start), opening square bracket (only when it can be interpreted as 1399 introducing a POSIX class name, or for a special compatibility feature - see 1400 the next two sections), and the terminating closing square bracket. However, 1401 escaping other non-alphanumeric characters does no harm. 1402 </P> 1403 <br><a name="SEC10" href="#TOC1">POSIX CHARACTER CLASSES</a><br> 1404 <P> 1405 Perl supports the POSIX notation for character classes. This uses names 1406 enclosed by [: and :] within the enclosing square brackets. PCRE2 also supports 1407 this notation. For example, 1408 <pre> 1409 [01[:alpha:]%] 1410 </pre> 1411 matches "0", "1", any alphabetic character, or "%". The supported class names 1412 are: 1413 <pre> 1414 alnum letters and digits 1415 alpha letters 1416 ascii character codes 0 - 127 1417 blank space or tab only 1418 cntrl control characters 1419 digit decimal digits (same as \d) 1420 graph printing characters, excluding space 1421 lower lower case letters 1422 print printing characters, including space 1423 punct printing characters, excluding letters and digits and space 1424 space white space (the same as \s from PCRE2 8.34) 1425 upper upper case letters 1426 word "word" characters (same as \w) 1427 xdigit hexadecimal digits 1428 </pre> 1429 The default "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), 1430 and space (32). If locale-specific matching is taking place, the list of space 1431 characters may be different; there may be fewer or more of them. "Space" and 1432 \s match the same set of characters. 1433 </P> 1434 <P> 1435 The name "word" is a Perl extension, and "blank" is a GNU extension from Perl 1436 5.8. Another Perl extension is negation, which is indicated by a ^ character 1437 after the colon. For example, 1438 <pre> 1439 [12[:^digit:]] 1440 </pre> 1441 matches "1", "2", or any non-digit. PCRE2 (and Perl) also recognize the POSIX 1442 syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not 1443 supported, and an error is given if they are encountered. 1444 </P> 1445 <P> 1446 By default, characters with values greater than 127 do not match any of the 1447 POSIX character classes, although this may be different for characters in the 1448 range 128-255 when locale-specific matching is happening. However, if the 1449 PCRE2_UCP option is passed to <b>pcre2_compile()</b>, some of the classes are 1450 changed so that Unicode character properties are used. This is achieved by 1451 replacing certain POSIX classes with other sequences, as follows: 1452 <pre> 1453 [:alnum:] becomes \p{Xan} 1454 [:alpha:] becomes \p{L} 1455 [:blank:] becomes \h 1456 [:cntrl:] becomes \p{Cc} 1457 [:digit:] becomes \p{Nd} 1458 [:lower:] becomes \p{Ll} 1459 [:space:] becomes \p{Xps} 1460 [:upper:] becomes \p{Lu} 1461 [:word:] becomes \p{Xwd} 1462 </pre> 1463 Negated versions, such as [:^alpha:] use \P instead of \p. Three other POSIX 1464 classes are handled specially in UCP mode: 1465 </P> 1466 <P> 1467 [:graph:] 1468 This matches characters that have glyphs that mark the page when printed. In 1469 Unicode property terms, it matches all characters with the L, M, N, P, S, or Cf 1470 properties, except for: 1471 <pre> 1472 U+061C Arabic Letter Mark 1473 U+180E Mongolian Vowel Separator 1474 U+2066 - U+2069 Various "isolate"s 1475 1476 </PRE> 1477 </P> 1478 <P> 1479 [:print:] 1480 This matches the same characters as [:graph:] plus space characters that are 1481 not controls, that is, characters with the Zs property. 1482 </P> 1483 <P> 1484 [:punct:] 1485 This matches all characters that have the Unicode P (punctuation) property, 1486 plus those characters with code points less than 256 that have the S (Symbol) 1487 property. 1488 </P> 1489 <P> 1490 The other POSIX classes are unchanged, and match only characters with code 1491 points less than 256. 1492 </P> 1493 <br><a name="SEC11" href="#TOC1">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a><br> 1494 <P> 1495 In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly 1496 syntax [[:<:]] and [[:>:]] is used for matching "start of word" and "end of 1497 word". PCRE2 treats these items as follows: 1498 <pre> 1499 [[:<:]] is converted to \b(?=\w) 1500 [[:>:]] is converted to \b(?<=\w) 1501 </pre> 1502 Only these exact character sequences are recognized. A sequence such as 1503 [a[:<:]b] provokes error for an unrecognized POSIX class name. This support is 1504 not compatible with Perl. It is provided to help migrations from other 1505 environments, and is best not used in any new patterns. Note that \b matches 1506 at the start and the end of a word (see 1507 <a href="#smallassertions">"Simple assertions"</a> 1508 above), and in a Perl-style pattern the preceding or following character 1509 normally shows which is wanted, without the need for the assertions that are 1510 used above in order to give exactly the POSIX behaviour. 1511 </P> 1512 <br><a name="SEC12" href="#TOC1">VERTICAL BAR</a><br> 1513 <P> 1514 Vertical bar characters are used to separate alternative patterns. For example, 1515 the pattern 1516 <pre> 1517 gilbert|sullivan 1518 </pre> 1519 matches either "gilbert" or "sullivan". Any number of alternatives may appear, 1520 and an empty alternative is permitted (matching the empty string). The matching 1521 process tries each alternative in turn, from left to right, and the first one 1522 that succeeds is used. If the alternatives are within a subpattern 1523 <a href="#subpattern">(defined below),</a> 1524 "succeeds" means matching the rest of the main pattern as well as the 1525 alternative in the subpattern. 1526 </P> 1527 <br><a name="SEC13" href="#TOC1">INTERNAL OPTION SETTING</a><br> 1528 <P> 1529 The settings of the PCRE2_CASELESS, PCRE2_MULTILINE, PCRE2_DOTALL, and 1530 PCRE2_EXTENDED options (which are Perl-compatible) can be changed from within 1531 the pattern by a sequence of Perl option letters enclosed between "(?" and ")". 1532 The option letters are 1533 <pre> 1534 i for PCRE2_CASELESS 1535 m for PCRE2_MULTILINE 1536 s for PCRE2_DOTALL 1537 x for PCRE2_EXTENDED 1538 </pre> 1539 For example, (?im) sets caseless, multiline matching. It is also possible to 1540 unset these options by preceding the letter with a hyphen, and a combined 1541 setting and unsetting such as (?im-sx), which sets PCRE2_CASELESS and 1542 PCRE2_MULTILINE while unsetting PCRE2_DOTALL and PCRE2_EXTENDED, is also 1543 permitted. If a letter appears both before and after the hyphen, the option is 1544 unset. An empty options setting "(?)" is allowed. Needless to say, it has no 1545 effect. 1546 </P> 1547 <P> 1548 The PCRE2-specific options PCRE2_DUPNAMES and PCRE2_UNGREEDY can be changed in 1549 the same way as the Perl-compatible options by using the characters J and U 1550 respectively. 1551 </P> 1552 <P> 1553 When one of these option changes occurs at top level (that is, not inside 1554 subpattern parentheses), the change applies to the remainder of the pattern 1555 that follows. If the change is placed right at the start of a pattern, PCRE2 1556 extracts it into the global options (and it will therefore show up in data 1557 extracted by the <b>pcre2_pattern_info()</b> function). 1558 </P> 1559 <P> 1560 An option change within a subpattern (see below for a description of 1561 subpatterns) affects only that part of the subpattern that follows it, so 1562 <pre> 1563 (a(?i)b)c 1564 </pre> 1565 matches abc and aBc and no other strings (assuming PCRE2_CASELESS is not used). 1566 By this means, options can be made to have different settings in different 1567 parts of the pattern. Any changes made in one alternative do carry on 1568 into subsequent branches within the same subpattern. For example, 1569 <pre> 1570 (a(?i)b|c) 1571 </pre> 1572 matches "ab", "aB", "c", and "C", even though when matching "C" the first 1573 branch is abandoned before the option setting. This is because the effects of 1574 option settings happen at compile time. There would be some very weird 1575 behaviour otherwise. 1576 </P> 1577 <P> 1578 As a convenient shorthand, if any option settings are required at the start of 1579 a non-capturing subpattern (see the next section), the option letters may 1580 appear between the "?" and the ":". Thus the two patterns 1581 <pre> 1582 (?i:saturday|sunday) 1583 (?:(?i)saturday|sunday) 1584 </pre> 1585 match exactly the same set of strings. 1586 </P> 1587 <P> 1588 <b>Note:</b> There are other PCRE2-specific options that can be set by the 1589 application when the compiling function is called. The pattern can contain 1590 special leading sequences such as (*CRLF) to override what the application has 1591 set or what has been defaulted. Details are given in the section entitled 1592 <a href="#newlineseq">"Newline sequences"</a> 1593 above. There are also the (*UTF) and (*UCP) leading sequences that can be used 1594 to set UTF and Unicode property modes; they are equivalent to setting the 1595 PCRE2_UTF and PCRE2_UCP options, respectively. However, the application can set 1596 the PCRE2_NEVER_UTF and PCRE2_NEVER_UCP options, which lock out the use of the 1597 (*UTF) and (*UCP) sequences. 1598 <a name="subpattern"></a></P> 1599 <br><a name="SEC14" href="#TOC1">SUBPATTERNS</a><br> 1600 <P> 1601 Subpatterns are delimited by parentheses (round brackets), which can be nested. 1602 Turning part of a pattern into a subpattern does two things: 1603 <br> 1604 <br> 1605 1. It localizes a set of alternatives. For example, the pattern 1606 <pre> 1607 cat(aract|erpillar|) 1608 </pre> 1609 matches "cataract", "caterpillar", or "cat". Without the parentheses, it would 1610 match "cataract", "erpillar" or an empty string. 1611 <br> 1612 <br> 1613 2. It sets up the subpattern as a capturing subpattern. This means that, when 1614 the whole pattern matches, the portion of the subject string that matched the 1615 subpattern is passed back to the caller, separately from the portion that 1616 matched the whole pattern. (This applies only to the traditional matching 1617 function; the DFA matching function does not support capturing.) 1618 </P> 1619 <P> 1620 Opening parentheses are counted from left to right (starting from 1) to obtain 1621 numbers for the capturing subpatterns. For example, if the string "the red 1622 king" is matched against the pattern 1623 <pre> 1624 the ((red|white) (king|queen)) 1625 </pre> 1626 the captured substrings are "red king", "red", and "king", and are numbered 1, 1627 2, and 3, respectively. 1628 </P> 1629 <P> 1630 The fact that plain parentheses fulfil two functions is not always helpful. 1631 There are often times when a grouping subpattern is required without a 1632 capturing requirement. If an opening parenthesis is followed by a question mark 1633 and a colon, the subpattern does not do any capturing, and is not counted when 1634 computing the number of any subsequent capturing subpatterns. For example, if 1635 the string "the white queen" is matched against the pattern 1636 <pre> 1637 the ((?:red|white) (king|queen)) 1638 </pre> 1639 the captured substrings are "white queen" and "queen", and are numbered 1 and 1640 2. The maximum number of capturing subpatterns is 65535. 1641 </P> 1642 <P> 1643 As a convenient shorthand, if any option settings are required at the start of 1644 a non-capturing subpattern, the option letters may appear between the "?" and 1645 the ":". Thus the two patterns 1646 <pre> 1647 (?i:saturday|sunday) 1648 (?:(?i)saturday|sunday) 1649 </pre> 1650 match exactly the same set of strings. Because alternative branches are tried 1651 from left to right, and options are not reset until the end of the subpattern 1652 is reached, an option setting in one branch does affect subsequent branches, so 1653 the above patterns match "SUNDAY" as well as "Saturday". 1654 <a name="dupsubpatternnumber"></a></P> 1655 <br><a name="SEC15" href="#TOC1">DUPLICATE SUBPATTERN NUMBERS</a><br> 1656 <P> 1657 Perl 5.10 introduced a feature whereby each alternative in a subpattern uses 1658 the same numbers for its capturing parentheses. Such a subpattern starts with 1659 (?| and is itself a non-capturing subpattern. For example, consider this 1660 pattern: 1661 <pre> 1662 (?|(Sat)ur|(Sun))day 1663 </pre> 1664 Because the two alternatives are inside a (?| group, both sets of capturing 1665 parentheses are numbered one. Thus, when the pattern matches, you can look 1666 at captured substring number one, whichever alternative matched. This construct 1667 is useful when you want to capture part, but not all, of one of a number of 1668 alternatives. Inside a (?| group, parentheses are numbered as usual, but the 1669 number is reset at the start of each branch. The numbers of any capturing 1670 parentheses that follow the subpattern start after the highest number used in 1671 any branch. The following example is taken from the Perl documentation. The 1672 numbers underneath show in which buffer the captured content will be stored. 1673 <pre> 1674 # before ---------------branch-reset----------- after 1675 / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x 1676 # 1 2 2 3 2 3 4 1677 </pre> 1678 A back reference to a numbered subpattern uses the most recent value that is 1679 set for that number by any subpattern. The following pattern matches "abcabc" 1680 or "defdef": 1681 <pre> 1682 /(?|(abc)|(def))\1/ 1683 </pre> 1684 In contrast, a subroutine call to a numbered subpattern always refers to the 1685 first one in the pattern with the given number. The following pattern matches 1686 "abcabc" or "defabc": 1687 <pre> 1688 /(?|(abc)|(def))(?1)/ 1689 </pre> 1690 A relative reference such as (?-1) is no different: it is just a convenient way 1691 of computing an absolute group number. 1692 </P> 1693 <P> 1694 If a 1695 <a href="#conditions">condition test</a> 1696 for a subpattern's having matched refers to a non-unique number, the test is 1697 true if any of the subpatterns of that number have matched. 1698 </P> 1699 <P> 1700 An alternative approach to using this "branch reset" feature is to use 1701 duplicate named subpatterns, as described in the next section. 1702 </P> 1703 <br><a name="SEC16" href="#TOC1">NAMED SUBPATTERNS</a><br> 1704 <P> 1705 Identifying capturing parentheses by number is simple, but it can be very hard 1706 to keep track of the numbers in complicated regular expressions. Furthermore, 1707 if an expression is modified, the numbers may change. To help with this 1708 difficulty, PCRE2 supports the naming of subpatterns. This feature was not 1709 added to Perl until release 5.10. Python had the feature earlier, and PCRE1 1710 introduced it at release 4.0, using the Python syntax. PCRE2 supports both the 1711 Perl and the Python syntax. Perl allows identically numbered subpatterns to 1712 have different names, but PCRE2 does not. 1713 </P> 1714 <P> 1715 In PCRE2, a subpattern can be named in one of three ways: (?<name>...) or 1716 (?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing 1717 parentheses from other parts of the pattern, such as 1718 <a href="#backreferences">back references,</a> 1719 <a href="#recursion">recursion,</a> 1720 and 1721 <a href="#conditions">conditions,</a> 1722 can be made by name as well as by number. 1723 </P> 1724 <P> 1725 Names consist of up to 32 alphanumeric characters and underscores, but must 1726 start with a non-digit. Named capturing parentheses are still allocated numbers 1727 as well as names, exactly as if the names were not present. The PCRE2 API 1728 provides function calls for extracting the name-to-number translation table 1729 from a compiled pattern. There are also convenience functions for extracting a 1730 captured substring by name. 1731 </P> 1732 <P> 1733 By default, a name must be unique within a pattern, but it is possible to relax 1734 this constraint by setting the PCRE2_DUPNAMES option at compile time. 1735 (Duplicate names are also always permitted for subpatterns with the same 1736 number, set up as described in the previous section.) Duplicate names can be 1737 useful for patterns where only one instance of the named parentheses can match. 1738 Suppose you want to match the name of a weekday, either as a 3-letter 1739 abbreviation or as the full name, and in both cases you want to extract the 1740 abbreviation. This pattern (ignoring the line breaks) does the job: 1741 <pre> 1742 (?<DN>Mon|Fri|Sun)(?:day)?| 1743 (?<DN>Tue)(?:sday)?| 1744 (?<DN>Wed)(?:nesday)?| 1745 (?<DN>Thu)(?:rsday)?| 1746 (?<DN>Sat)(?:urday)? 1747 </pre> 1748 There are five capturing substrings, but only one is ever set after a match. 1749 (An alternative way of solving this problem is to use a "branch reset" 1750 subpattern, as described in the previous section.) 1751 </P> 1752 <P> 1753 The convenience functions for extracting the data by name returns the substring 1754 for the first (and in this example, the only) subpattern of that name that 1755 matched. This saves searching to find which numbered subpattern it was. 1756 </P> 1757 <P> 1758 If you make a back reference to a non-unique named subpattern from elsewhere in 1759 the pattern, the subpatterns to which the name refers are checked in the order 1760 in which they appear in the overall pattern. The first one that is set is used 1761 for the reference. For example, this pattern matches both "foofoo" and 1762 "barbar" but not "foobar" or "barfoo": 1763 <pre> 1764 (?:(?<n>foo)|(?<n>bar))\k<n> 1765 1766 </PRE> 1767 </P> 1768 <P> 1769 If you make a subroutine call to a non-unique named subpattern, the one that 1770 corresponds to the first occurrence of the name is used. In the absence of 1771 duplicate numbers (see the previous section) this is the one with the lowest 1772 number. 1773 </P> 1774 <P> 1775 If you use a named reference in a condition 1776 test (see the 1777 <a href="#conditions">section about conditions</a> 1778 below), either to check whether a subpattern has matched, or to check for 1779 recursion, all subpatterns with the same name are tested. If the condition is 1780 true for any one of them, the overall condition is true. This is the same 1781 behaviour as testing by number. For further details of the interfaces for 1782 handling named subpatterns, see the 1783 <a href="pcre2api.html"><b>pcre2api</b></a> 1784 documentation. 1785 </P> 1786 <P> 1787 <b>Warning:</b> You cannot use different names to distinguish between two 1788 subpatterns with the same number because PCRE2 uses only the numbers when 1789 matching. For this reason, an error is given at compile time if different names 1790 are given to subpatterns with the same number. However, you can always give the 1791 same name to subpatterns with the same number, even when PCRE2_DUPNAMES is not 1792 set. 1793 </P> 1794 <br><a name="SEC17" href="#TOC1">REPETITION</a><br> 1795 <P> 1796 Repetition is specified by quantifiers, which can follow any of the following 1797 items: 1798 <pre> 1799 a literal data character 1800 the dot metacharacter 1801 the \C escape sequence 1802 the \X escape sequence 1803 the \R escape sequence 1804 an escape such as \d or \pL that matches a single character 1805 a character class 1806 a back reference 1807 a parenthesized subpattern (including most assertions) 1808 a subroutine call to a subpattern (recursive or otherwise) 1809 </pre> 1810 The general repetition quantifier specifies a minimum and maximum number of 1811 permitted matches, by giving the two numbers in curly brackets (braces), 1812 separated by a comma. The numbers must be less than 65536, and the first must 1813 be less than or equal to the second. For example: 1814 <pre> 1815 z{2,4} 1816 </pre> 1817 matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special 1818 character. If the second number is omitted, but the comma is present, there is 1819 no upper limit; if the second number and the comma are both omitted, the 1820 quantifier specifies an exact number of required matches. Thus 1821 <pre> 1822 [aeiou]{3,} 1823 </pre> 1824 matches at least 3 successive vowels, but may match many more, whereas 1825 <pre> 1826 \d{8} 1827 </pre> 1828 matches exactly 8 digits. An opening curly bracket that appears in a position 1829 where a quantifier is not allowed, or one that does not match the syntax of a 1830 quantifier, is taken as a literal character. For example, {,6} is not a 1831 quantifier, but a literal string of four characters. 1832 </P> 1833 <P> 1834 In UTF modes, quantifiers apply to characters rather than to individual code 1835 units. Thus, for example, \x{100}{2} matches two characters, each of 1836 which is represented by a two-byte sequence in a UTF-8 string. Similarly, 1837 \X{3} matches three Unicode extended grapheme clusters, each of which may be 1838 several code units long (and they may be of different lengths). 1839 </P> 1840 <P> 1841 The quantifier {0} is permitted, causing the expression to behave as if the 1842 previous item and the quantifier were not present. This may be useful for 1843 subpatterns that are referenced as 1844 <a href="#subpatternsassubroutines">subroutines</a> 1845 from elsewhere in the pattern (but see also the section entitled 1846 <a href="#subdefine">"Defining subpatterns for use by reference only"</a> 1847 below). Items other than subpatterns that have a {0} quantifier are omitted 1848 from the compiled pattern. 1849 </P> 1850 <P> 1851 For convenience, the three most common quantifiers have single-character 1852 abbreviations: 1853 <pre> 1854 * is equivalent to {0,} 1855 + is equivalent to {1,} 1856 ? is equivalent to {0,1} 1857 </pre> 1858 It is possible to construct infinite loops by following a subpattern that can 1859 match no characters with a quantifier that has no upper limit, for example: 1860 <pre> 1861 (a?)* 1862 </pre> 1863 Earlier versions of Perl and PCRE1 used to give an error at compile time for 1864 such patterns. However, because there are cases where this can be useful, such 1865 patterns are now accepted, but if any repetition of the subpattern does in fact 1866 match no characters, the loop is forcibly broken. 1867 </P> 1868 <P> 1869 By default, the quantifiers are "greedy", that is, they match as much as 1870 possible (up to the maximum number of permitted times), without causing the 1871 rest of the pattern to fail. The classic example of where this gives problems 1872 is in trying to match comments in C programs. These appear between /* and */ 1873 and within the comment, individual * and / characters may appear. An attempt to 1874 match C comments by applying the pattern 1875 <pre> 1876 /\*.*\*/ 1877 </pre> 1878 to the string 1879 <pre> 1880 /* first comment */ not comment /* second comment */ 1881 </pre> 1882 fails, because it matches the entire string owing to the greediness of the .* 1883 item. 1884 </P> 1885 <P> 1886 If a quantifier is followed by a question mark, it ceases to be greedy, and 1887 instead matches the minimum number of times possible, so the pattern 1888 <pre> 1889 /\*.*?\*/ 1890 </pre> 1891 does the right thing with the C comments. The meaning of the various 1892 quantifiers is not otherwise changed, just the preferred number of matches. 1893 Do not confuse this use of question mark with its use as a quantifier in its 1894 own right. Because it has two uses, it can sometimes appear doubled, as in 1895 <pre> 1896 \d??\d 1897 </pre> 1898 which matches one digit by preference, but can match two if that is the only 1899 way the rest of the pattern matches. 1900 </P> 1901 <P> 1902 If the PCRE2_UNGREEDY option is set (an option that is not available in Perl), 1903 the quantifiers are not greedy by default, but individual ones can be made 1904 greedy by following them with a question mark. In other words, it inverts the 1905 default behaviour. 1906 </P> 1907 <P> 1908 When a parenthesized subpattern is quantified with a minimum repeat count that 1909 is greater than 1 or with a limited maximum, more memory is required for the 1910 compiled pattern, in proportion to the size of the minimum or maximum. 1911 </P> 1912 <P> 1913 If a pattern starts with .* or .{0,} and the PCRE2_DOTALL option (equivalent 1914 to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is 1915 implicitly anchored, because whatever follows will be tried against every 1916 character position in the subject string, so there is no point in retrying the 1917 overall match at any position after the first. PCRE2 normally treats such a 1918 pattern as though it were preceded by \A. 1919 </P> 1920 <P> 1921 In cases where it is known that the subject string contains no newlines, it is 1922 worth setting PCRE2_DOTALL in order to obtain this optimization, or 1923 alternatively, using ^ to indicate anchoring explicitly. 1924 </P> 1925 <P> 1926 However, there are some cases where the optimization cannot be used. When .* 1927 is inside capturing parentheses that are the subject of a back reference 1928 elsewhere in the pattern, a match at the start may fail where a later one 1929 succeeds. Consider, for example: 1930 <pre> 1931 (.*)abc\1 1932 </pre> 1933 If the subject is "xyz123abc123" the match point is the fourth character. For 1934 this reason, such a pattern is not implicitly anchored. 1935 </P> 1936 <P> 1937 Another case where implicit anchoring is not applied is when the leading .* is 1938 inside an atomic group. Once again, a match at the start may fail where a later 1939 one succeeds. Consider this pattern: 1940 <pre> 1941 (?>.*?a)b 1942 </pre> 1943 It matches "ab" in the subject "aab". The use of the backtracking control verbs 1944 (*PRUNE) and (*SKIP) also disable this optimization, and there is an option, 1945 PCRE2_NO_DOTSTAR_ANCHOR, to do so explicitly. 1946 </P> 1947 <P> 1948 When a capturing subpattern is repeated, the value captured is the substring 1949 that matched the final iteration. For example, after 1950 <pre> 1951 (tweedle[dume]{3}\s*)+ 1952 </pre> 1953 has matched "tweedledum tweedledee" the value of the captured substring is 1954 "tweedledee". However, if there are nested capturing subpatterns, the 1955 corresponding captured values may have been set in previous iterations. For 1956 example, after 1957 <pre> 1958 (a|(b))+ 1959 </pre> 1960 matches "aba" the value of the second captured substring is "b". 1961 <a name="atomicgroup"></a></P> 1962 <br><a name="SEC18" href="#TOC1">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a><br> 1963 <P> 1964 With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") 1965 repetition, failure of what follows normally causes the repeated item to be 1966 re-evaluated to see if a different number of repeats allows the rest of the 1967 pattern to match. Sometimes it is useful to prevent this, either to change the 1968 nature of the match, or to cause it fail earlier than it otherwise might, when 1969 the author of the pattern knows there is no point in carrying on. 1970 </P> 1971 <P> 1972 Consider, for example, the pattern \d+foo when applied to the subject line 1973 <pre> 1974 123456bar 1975 </pre> 1976 After matching all 6 digits and then failing to match "foo", the normal 1977 action of the matcher is to try again with only 5 digits matching the \d+ 1978 item, and then with 4, and so on, before ultimately failing. "Atomic grouping" 1979 (a term taken from Jeffrey Friedl's book) provides the means for specifying 1980 that once a subpattern has matched, it is not to be re-evaluated in this way. 1981 </P> 1982 <P> 1983 If we use atomic grouping for the previous example, the matcher gives up 1984 immediately on failing to match "foo" the first time. The notation is a kind of 1985 special parenthesis, starting with (?> as in this example: 1986 <pre> 1987 (?>\d+)foo 1988 </pre> 1989 This kind of parenthesis "locks up" the part of the pattern it contains once 1990 it has matched, and a failure further into the pattern is prevented from 1991 backtracking into it. Backtracking past it to previous items, however, works as 1992 normal. 1993 </P> 1994 <P> 1995 An alternative description is that a subpattern of this type matches exactly 1996 the string of characters that an identical standalone pattern would match, if 1997 anchored at the current point in the subject string. 1998 </P> 1999 <P> 2000 Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as 2001 the above example can be thought of as a maximizing repeat that must swallow 2002 everything it can. So, while both \d+ and \d+? are prepared to adjust the 2003 number of digits they match in order to make the rest of the pattern match, 2004 (?>\d+) can only match an entire sequence of digits. 2005 </P> 2006 <P> 2007 Atomic groups in general can of course contain arbitrarily complicated 2008 subpatterns, and can be nested. However, when the subpattern for an atomic 2009 group is just a single repeated item, as in the example above, a simpler 2010 notation, called a "possessive quantifier" can be used. This consists of an 2011 additional + character following a quantifier. Using this notation, the 2012 previous example can be rewritten as 2013 <pre> 2014 \d++foo 2015 </pre> 2016 Note that a possessive quantifier can be used with an entire group, for 2017 example: 2018 <pre> 2019 (abc|xyz){2,3}+ 2020 </pre> 2021 Possessive quantifiers are always greedy; the setting of the PCRE2_UNGREEDY 2022 option is ignored. They are a convenient notation for the simpler forms of 2023 atomic group. However, there is no difference in the meaning of a possessive 2024 quantifier and the equivalent atomic group, though there may be a performance 2025 difference; possessive quantifiers should be slightly faster. 2026 </P> 2027 <P> 2028 The possessive quantifier syntax is an extension to the Perl 5.8 syntax. 2029 Jeffrey Friedl originated the idea (and the name) in the first edition of his 2030 book. Mike McCloskey liked it, so implemented it when he built Sun's Java 2031 package, and PCRE1 copied it from there. It ultimately found its way into Perl 2032 at release 5.10. 2033 </P> 2034 <P> 2035 PCRE2 has an optimization that automatically "possessifies" certain simple 2036 pattern constructs. For example, the sequence A+B is treated as A++B because 2037 there is no point in backtracking into a sequence of A's when B must follow. 2038 This feature can be disabled by the PCRE2_NO_AUTOPOSSESS option, or starting 2039 the pattern with (*NO_AUTO_POSSESS). 2040 </P> 2041 <P> 2042 When a pattern contains an unlimited repeat inside a subpattern that can itself 2043 be repeated an unlimited number of times, the use of an atomic group is the 2044 only way to avoid some failing matches taking a very long time indeed. The 2045 pattern 2046 <pre> 2047 (\D+|<\d+>)*[!?] 2048 </pre> 2049 matches an unlimited number of substrings that either consist of non-digits, or 2050 digits enclosed in <>, followed by either ! or ?. When it matches, it runs 2051 quickly. However, if it is applied to 2052 <pre> 2053 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 2054 </pre> 2055 it takes a long time before reporting failure. This is because the string can 2056 be divided between the internal \D+ repeat and the external * repeat in a 2057 large number of ways, and all have to be tried. (The example uses [!?] rather 2058 than a single character at the end, because both PCRE2 and Perl have an 2059 optimization that allows for fast failure when a single character is used. They 2060 remember the last single character that is required for a match, and fail early 2061 if it is not present in the string.) If the pattern is changed so that it uses 2062 an atomic group, like this: 2063 <pre> 2064 ((?>\D+)|<\d+>)*[!?] 2065 </pre> 2066 sequences of non-digits cannot be broken, and failure happens quickly. 2067 <a name="backreferences"></a></P> 2068 <br><a name="SEC19" href="#TOC1">BACK REFERENCES</a><br> 2069 <P> 2070 Outside a character class, a backslash followed by a digit greater than 0 (and 2071 possibly further digits) is a back reference to a capturing subpattern earlier 2072 (that is, to its left) in the pattern, provided there have been that many 2073 previous capturing left parentheses. 2074 </P> 2075 <P> 2076 However, if the decimal number following the backslash is less than 8, it is 2077 always taken as a back reference, and causes an error only if there are not 2078 that many capturing left parentheses in the entire pattern. In other words, the 2079 parentheses that are referenced need not be to the left of the reference for 2080 numbers less than 8. A "forward back reference" of this type can make sense 2081 when a repetition is involved and the subpattern to the right has participated 2082 in an earlier iteration. 2083 </P> 2084 <P> 2085 It is not possible to have a numerical "forward back reference" to a subpattern 2086 whose number is 8 or more using this syntax because a sequence such as \50 is 2087 interpreted as a character defined in octal. See the subsection entitled 2088 "Non-printing characters" 2089 <a href="#digitsafterbackslash">above</a> 2090 for further details of the handling of digits following a backslash. There is 2091 no such problem when named parentheses are used. A back reference to any 2092 subpattern is possible using named parentheses (see below). 2093 </P> 2094 <P> 2095 Another way of avoiding the ambiguity inherent in the use of digits following a 2096 backslash is to use the \g escape sequence. This escape must be followed by an 2097 unsigned number or a negative number, optionally enclosed in braces. These 2098 examples are all identical: 2099 <pre> 2100 (ring), \1 2101 (ring), \g1 2102 (ring), \g{1} 2103 </pre> 2104 An unsigned number specifies an absolute reference without the ambiguity that 2105 is present in the older syntax. It is also useful when literal digits follow 2106 the reference. A negative number is a relative reference. Consider this 2107 example: 2108 <pre> 2109 (abc(def)ghi)\g{-1} 2110 </pre> 2111 The sequence \g{-1} is a reference to the most recently started capturing 2112 subpattern before \g, that is, is it equivalent to \2 in this example. 2113 Similarly, \g{-2} would be equivalent to \1. The use of relative references 2114 can be helpful in long patterns, and also in patterns that are created by 2115 joining together fragments that contain references within themselves. 2116 </P> 2117 <P> 2118 A back reference matches whatever actually matched the capturing subpattern in 2119 the current subject string, rather than anything matching the subpattern 2120 itself (see 2121 <a href="#subpatternsassubroutines">"Subpatterns as subroutines"</a> 2122 below for a way of doing that). So the pattern 2123 <pre> 2124 (sens|respons)e and \1ibility 2125 </pre> 2126 matches "sense and sensibility" and "response and responsibility", but not 2127 "sense and responsibility". If caseful matching is in force at the time of the 2128 back reference, the case of letters is relevant. For example, 2129 <pre> 2130 ((?i)rah)\s+\1 2131 </pre> 2132 matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original 2133 capturing subpattern is matched caselessly. 2134 </P> 2135 <P> 2136 There are several different ways of writing back references to named 2137 subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or 2138 \k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified 2139 back reference syntax, in which \g can be used for both numeric and named 2140 references, is also supported. We could rewrite the above example in any of 2141 the following ways: 2142 <pre> 2143 (?<p1>(?i)rah)\s+\k<p1> 2144 (?'p1'(?i)rah)\s+\k{p1} 2145 (?P<p1>(?i)rah)\s+(?P=p1) 2146 (?<p1>(?i)rah)\s+\g{p1} 2147 </pre> 2148 A subpattern that is referenced by name may appear in the pattern before or 2149 after the reference. 2150 </P> 2151 <P> 2152 There may be more than one back reference to the same subpattern. If a 2153 subpattern has not actually been used in a particular match, any back 2154 references to it always fail by default. For example, the pattern 2155 <pre> 2156 (a|(bc))\2 2157 </pre> 2158 always fails if it starts to match "a" rather than "bc". However, if the 2159 PCRE2_MATCH_UNSET_BACKREF option is set at compile time, a back reference to an 2160 unset value matches an empty string. 2161 </P> 2162 <P> 2163 Because there may be many capturing parentheses in a pattern, all digits 2164 following a backslash are taken as part of a potential back reference number. 2165 If the pattern continues with a digit character, some delimiter must be used to 2166 terminate the back reference. If the PCRE2_EXTENDED option is set, this can be 2167 white space. Otherwise, the \g{ syntax or an empty comment (see 2168 <a href="#comments">"Comments"</a> 2169 below) can be used. 2170 </P> 2171 <br><b> 2172 Recursive back references 2173 </b><br> 2174 <P> 2175 A back reference that occurs inside the parentheses to which it refers fails 2176 when the subpattern is first used, so, for example, (a\1) never matches. 2177 However, such references can be useful inside repeated subpatterns. For 2178 example, the pattern 2179 <pre> 2180 (a|b\1)+ 2181 </pre> 2182 matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of 2183 the subpattern, the back reference matches the character string corresponding 2184 to the previous iteration. In order for this to work, the pattern must be such 2185 that the first iteration does not need to match the back reference. This can be 2186 done using alternation, as in the example above, or by a quantifier with a 2187 minimum of zero. 2188 </P> 2189 <P> 2190 Back references of this type cause the group that they reference to be treated 2191 as an 2192 <a href="#atomicgroup">atomic group.</a> 2193 Once the whole group has been matched, a subsequent matching failure cannot 2194 cause backtracking into the middle of the group. 2195 <a name="bigassertions"></a></P> 2196 <br><a name="SEC20" href="#TOC1">ASSERTIONS</a><br> 2197 <P> 2198 An assertion is a test on the characters following or preceding the current 2199 matching point that does not consume any characters. The simple assertions 2200 coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described 2201 <a href="#smallassertions">above.</a> 2202 </P> 2203 <P> 2204 More complicated assertions are coded as subpatterns. There are two kinds: 2205 those that look ahead of the current position in the subject string, and those 2206 that look behind it. An assertion subpattern is matched in the normal way, 2207 except that it does not cause the current matching position to be changed. 2208 </P> 2209 <P> 2210 Assertion subpatterns are not capturing subpatterns. If such an assertion 2211 contains capturing subpatterns within it, these are counted for the purposes of 2212 numbering the capturing subpatterns in the whole pattern. However, substring 2213 capturing is carried out only for positive assertions. (Perl sometimes, but not 2214 always, does do capturing in negative assertions.) 2215 </P> 2216 <P> 2217 For compatibility with Perl, most assertion subpatterns may be repeated; though 2218 it makes no sense to assert the same thing several times, the side effect of 2219 capturing parentheses may occasionally be useful. However, an assertion that 2220 forms the condition for a conditional subpattern may not be quantified. In 2221 practice, for other assertions, there only three cases: 2222 <br> 2223 <br> 2224 (1) If the quantifier is {0}, the assertion is never obeyed during matching. 2225 However, it may contain internal capturing parenthesized groups that are called 2226 from elsewhere via the 2227 <a href="#subpatternsassubroutines">subroutine mechanism.</a> 2228 <br> 2229 <br> 2230 (2) If quantifier is {0,n} where n is greater than zero, it is treated as if it 2231 were {0,1}. At run time, the rest of the pattern match is tried with and 2232 without the assertion, the order depending on the greediness of the quantifier. 2233 <br> 2234 <br> 2235 (3) If the minimum repetition is greater than zero, the quantifier is ignored. 2236 The assertion is obeyed just once when encountered during matching. 2237 </P> 2238 <br><b> 2239 Lookahead assertions 2240 </b><br> 2241 <P> 2242 Lookahead assertions start with (?= for positive assertions and (?! for 2243 negative assertions. For example, 2244 <pre> 2245 \w+(?=;) 2246 </pre> 2247 matches a word followed by a semicolon, but does not include the semicolon in 2248 the match, and 2249 <pre> 2250 foo(?!bar) 2251 </pre> 2252 matches any occurrence of "foo" that is not followed by "bar". Note that the 2253 apparently similar pattern 2254 <pre> 2255 (?!foo)bar 2256 </pre> 2257 does not find an occurrence of "bar" that is preceded by something other than 2258 "foo"; it finds any occurrence of "bar" whatsoever, because the assertion 2259 (?!foo) is always true when the next three characters are "bar". A 2260 lookbehind assertion is needed to achieve the other effect. 2261 </P> 2262 <P> 2263 If you want to force a matching failure at some point in a pattern, the most 2264 convenient way to do it is with (?!) because an empty string always matches, so 2265 an assertion that requires there not to be an empty string must always fail. 2266 The backtracking control verb (*FAIL) or (*F) is a synonym for (?!). 2267 <a name="lookbehind"></a></P> 2268 <br><b> 2269 Lookbehind assertions 2270 </b><br> 2271 <P> 2272 Lookbehind assertions start with (?<= for positive assertions and (?<! for 2273 negative assertions. For example, 2274 <pre> 2275 (?<!foo)bar 2276 </pre> 2277 does find an occurrence of "bar" that is not preceded by "foo". The contents of 2278 a lookbehind assertion are restricted such that all the strings it matches must 2279 have a fixed length. However, if there are several top-level alternatives, they 2280 do not all have to have the same fixed length. Thus 2281 <pre> 2282 (?<=bullock|donkey) 2283 </pre> 2284 is permitted, but 2285 <pre> 2286 (?<!dogs?|cats?) 2287 </pre> 2288 causes an error at compile time. Branches that match different length strings 2289 are permitted only at the top level of a lookbehind assertion. This is an 2290 extension compared with Perl, which requires all branches to match the same 2291 length of string. An assertion such as 2292 <pre> 2293 (?<=ab(c|de)) 2294 </pre> 2295 is not permitted, because its single top-level branch can match two different 2296 lengths, but it is acceptable to PCRE2 if rewritten to use two top-level 2297 branches: 2298 <pre> 2299 (?<=abc|abde) 2300 </pre> 2301 In some cases, the escape sequence \K 2302 <a href="#resetmatchstart">(see above)</a> 2303 can be used instead of a lookbehind assertion to get round the fixed-length 2304 restriction. 2305 </P> 2306 <P> 2307 The implementation of lookbehind assertions is, for each alternative, to 2308 temporarily move the current position back by the fixed length and then try to 2309 match. If there are insufficient characters before the current position, the 2310 assertion fails. 2311 </P> 2312 <P> 2313 In a UTF mode, PCRE2 does not allow the \C escape (which matches a single code 2314 unit even in a UTF mode) to appear in lookbehind assertions, because it makes 2315 it impossible to calculate the length of the lookbehind. The \X and \R 2316 escapes, which can match different numbers of code units, are also not 2317 permitted. 2318 </P> 2319 <P> 2320 <a href="#subpatternsassubroutines">"Subroutine"</a> 2321 calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long 2322 as the subpattern matches a fixed-length string. 2323 <a href="#recursion">Recursion,</a> 2324 however, is not supported. 2325 </P> 2326 <P> 2327 Possessive quantifiers can be used in conjunction with lookbehind assertions to 2328 specify efficient matching of fixed-length strings at the end of subject 2329 strings. Consider a simple pattern such as 2330 <pre> 2331 abcd$ 2332 </pre> 2333 when applied to a long string that does not match. Because matching proceeds 2334 from left to right, PCRE2 will look for each "a" in the subject and then see if 2335 what follows matches the rest of the pattern. If the pattern is specified as 2336 <pre> 2337 ^.*abcd$ 2338 </pre> 2339 the initial .* matches the entire string at first, but when this fails (because 2340 there is no following "a"), it backtracks to match all but the last character, 2341 then all but the last two characters, and so on. Once again the search for "a" 2342 covers the entire string, from right to left, so we are no better off. However, 2343 if the pattern is written as 2344 <pre> 2345 ^.*+(?<=abcd) 2346 </pre> 2347 there can be no backtracking for the .*+ item because of the possessive 2348 quantifier; it can match only the entire string. The subsequent lookbehind 2349 assertion does a single test on the last four characters. If it fails, the 2350 match fails immediately. For long strings, this approach makes a significant 2351 difference to the processing time. 2352 </P> 2353 <br><b> 2354 Using multiple assertions 2355 </b><br> 2356 <P> 2357 Several assertions (of any sort) may occur in succession. For example, 2358 <pre> 2359 (?<=\d{3})(?<!999)foo 2360 </pre> 2361 matches "foo" preceded by three digits that are not "999". Notice that each of 2362 the assertions is applied independently at the same point in the subject 2363 string. First there is a check that the previous three characters are all 2364 digits, and then there is a check that the same three characters are not "999". 2365 This pattern does <i>not</i> match "foo" preceded by six characters, the first 2366 of which are digits and the last three of which are not "999". For example, it 2367 doesn't match "123abcfoo". A pattern to do that is 2368 <pre> 2369 (?<=\d{3}...)(?<!999)foo 2370 </pre> 2371 This time the first assertion looks at the preceding six characters, checking 2372 that the first three are digits, and then the second assertion checks that the 2373 preceding three characters are not "999". 2374 </P> 2375 <P> 2376 Assertions can be nested in any combination. For example, 2377 <pre> 2378 (?<=(?<!foo)bar)baz 2379 </pre> 2380 matches an occurrence of "baz" that is preceded by "bar" which in turn is not 2381 preceded by "foo", while 2382 <pre> 2383 (?<=\d{3}(?!999)...)foo 2384 </pre> 2385 is another pattern that matches "foo" preceded by three digits and any three 2386 characters that are not "999". 2387 <a name="conditions"></a></P> 2388 <br><a name="SEC21" href="#TOC1">CONDITIONAL SUBPATTERNS</a><br> 2389 <P> 2390 It is possible to cause the matching process to obey a subpattern 2391 conditionally or to choose between two alternative subpatterns, depending on 2392 the result of an assertion, or whether a specific capturing subpattern has 2393 already been matched. The two possible forms of conditional subpattern are: 2394 <pre> 2395 (?(condition)yes-pattern) 2396 (?(condition)yes-pattern|no-pattern) 2397 </pre> 2398 If the condition is satisfied, the yes-pattern is used; otherwise the 2399 no-pattern (if present) is used. If there are more than two alternatives in the 2400 subpattern, a compile-time error occurs. Each of the two alternatives may 2401 itself contain nested subpatterns of any form, including conditional 2402 subpatterns; the restriction to two alternatives applies only at the level of 2403 the condition. This pattern fragment is an example where the alternatives are 2404 complex: 2405 <pre> 2406 (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) 2407 2408 </PRE> 2409 </P> 2410 <P> 2411 There are five kinds of condition: references to subpatterns, references to 2412 recursion, two pseudo-conditions called DEFINE and VERSION, and assertions. 2413 </P> 2414 <br><b> 2415 Checking for a used subpattern by number 2416 </b><br> 2417 <P> 2418 If the text between the parentheses consists of a sequence of digits, the 2419 condition is true if a capturing subpattern of that number has previously 2420 matched. If there is more than one capturing subpattern with the same number 2421 (see the earlier 2422 <a href="#recursion">section about duplicate subpattern numbers),</a> 2423 the condition is true if any of them have matched. An alternative notation is 2424 to precede the digits with a plus or minus sign. In this case, the subpattern 2425 number is relative rather than absolute. The most recently opened parentheses 2426 can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside 2427 loops it can also make sense to refer to subsequent groups. The next 2428 parentheses to be opened can be referenced as (?(+1), and so on. (The value 2429 zero in any of these forms is not used; it provokes a compile-time error.) 2430 </P> 2431 <P> 2432 Consider the following pattern, which contains non-significant white space to 2433 make it more readable (assume the PCRE2_EXTENDED option) and to divide it into 2434 three parts for ease of discussion: 2435 <pre> 2436 ( \( )? [^()]+ (?(1) \) ) 2437 </pre> 2438 The first part matches an optional opening parenthesis, and if that 2439 character is present, sets it as the first captured substring. The second part 2440 matches one or more characters that are not parentheses. The third part is a 2441 conditional subpattern that tests whether or not the first set of parentheses 2442 matched. If they did, that is, if subject started with an opening parenthesis, 2443 the condition is true, and so the yes-pattern is executed and a closing 2444 parenthesis is required. Otherwise, since no-pattern is not present, the 2445 subpattern matches nothing. In other words, this pattern matches a sequence of 2446 non-parentheses, optionally enclosed in parentheses. 2447 </P> 2448 <P> 2449 If you were embedding this pattern in a larger one, you could use a relative 2450 reference: 2451 <pre> 2452 ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... 2453 </pre> 2454 This makes the fragment independent of the parentheses in the larger pattern. 2455 </P> 2456 <br><b> 2457 Checking for a used subpattern by name 2458 </b><br> 2459 <P> 2460 Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used 2461 subpattern by name. For compatibility with earlier versions of PCRE1, which had 2462 this facility before Perl, the syntax (?(name)...) is also recognized. 2463 </P> 2464 <P> 2465 Rewriting the above example to use a named subpattern gives this: 2466 <pre> 2467 (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) 2468 </pre> 2469 If the name used in a condition of this kind is a duplicate, the test is 2470 applied to all subpatterns of the same name, and is true if any one of them has 2471 matched. 2472 </P> 2473 <br><b> 2474 Checking for pattern recursion 2475 </b><br> 2476 <P> 2477 If the condition is the string (R), and there is no subpattern with the name R, 2478 the condition is true if a recursive call to the whole pattern or any 2479 subpattern has been made. If digits or a name preceded by ampersand follow the 2480 letter R, for example: 2481 <pre> 2482 (?(R3)...) or (?(R&name)...) 2483 </pre> 2484 the condition is true if the most recent recursion is into a subpattern whose 2485 number or name is given. This condition does not check the entire recursion 2486 stack. If the name used in a condition of this kind is a duplicate, the test is 2487 applied to all subpatterns of the same name, and is true if any one of them is 2488 the most recent recursion. 2489 </P> 2490 <P> 2491 At "top level", all these recursion test conditions are false. 2492 <a href="#recursion">The syntax for recursive patterns</a> 2493 is described below. 2494 <a name="subdefine"></a></P> 2495 <br><b> 2496 Defining subpatterns for use by reference only 2497 </b><br> 2498 <P> 2499 If the condition is the string (DEFINE), and there is no subpattern with the 2500 name DEFINE, the condition is always false. In this case, there may be only one 2501 alternative in the subpattern. It is always skipped if control reaches this 2502 point in the pattern; the idea of DEFINE is that it can be used to define 2503 subroutines that can be referenced from elsewhere. (The use of 2504 <a href="#subpatternsassubroutines">subroutines</a> 2505 is described below.) For example, a pattern to match an IPv4 address such as 2506 "192.168.23.245" could be written like this (ignore white space and line 2507 breaks): 2508 <pre> 2509 (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) 2510 \b (?&byte) (\.(?&byte)){3} \b 2511 </pre> 2512 The first part of the pattern is a DEFINE group inside which a another group 2513 named "byte" is defined. This matches an individual component of an IPv4 2514 address (a number less than 256). When matching takes place, this part of the 2515 pattern is skipped because DEFINE acts like a false condition. The rest of the 2516 pattern uses references to the named group to match the four dot-separated 2517 components of an IPv4 address, insisting on a word boundary at each end. 2518 </P> 2519 <br><b> 2520 Checking the PCRE2 version 2521 </b><br> 2522 <P> 2523 Programs that link with a PCRE2 library can check the version by calling 2524 <b>pcre2_config()</b> with appropriate arguments. Users of applications that do 2525 not have access to the underlying code cannot do this. A special "condition" 2526 called VERSION exists to allow such users to discover which version of PCRE2 2527 they are dealing with by using this condition to match a string such as 2528 "yesno". VERSION must be followed either by "=" or ">=" and a version number. 2529 For example: 2530 <pre> 2531 (?(VERSION>=10.4)yes|no) 2532 </pre> 2533 This pattern matches "yes" if the PCRE2 version is greater or equal to 10.4, or 2534 "no" otherwise. The fractional part of the version number may not contain more 2535 than two digits. 2536 </P> 2537 <br><b> 2538 Assertion conditions 2539 </b><br> 2540 <P> 2541 If the condition is not in any of the above formats, it must be an assertion. 2542 This may be a positive or negative lookahead or lookbehind assertion. Consider 2543 this pattern, again containing non-significant white space, and with the two 2544 alternatives on the second line: 2545 <pre> 2546 (?(?=[^a-z]*[a-z]) 2547 \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) 2548 </pre> 2549 The condition is a positive lookahead assertion that matches an optional 2550 sequence of non-letters followed by a letter. In other words, it tests for the 2551 presence of at least one letter in the subject. If a letter is found, the 2552 subject is matched against the first alternative; otherwise it is matched 2553 against the second. This pattern matches strings in one of the two forms 2554 dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. 2555 <a name="comments"></a></P> 2556 <br><a name="SEC22" href="#TOC1">COMMENTS</a><br> 2557 <P> 2558 There are two ways of including comments in patterns that are processed by 2559 PCRE2. In both cases, the start of the comment must not be in a character 2560 class, nor in the middle of any other sequence of related characters such as 2561 (?: or a subpattern name or number. The characters that make up a comment play 2562 no part in the pattern matching. 2563 </P> 2564 <P> 2565 The sequence (?# marks the start of a comment that continues up to the next 2566 closing parenthesis. Nested parentheses are not permitted. If the 2567 PCRE2_EXTENDED option is set, an unescaped # character also introduces a 2568 comment, which in this case continues to immediately after the next newline 2569 character or character sequence in the pattern. Which characters are 2570 interpreted as newlines is controlled by an option passed to the compiling 2571 function or by a special sequence at the start of the pattern, as described in 2572 the section entitled 2573 <a href="#newlines">"Newline conventions"</a> 2574 above. Note that the end of this type of comment is a literal newline sequence 2575 in the pattern; escape sequences that happen to represent a newline do not 2576 count. For example, consider this pattern when PCRE2_EXTENDED is set, and the 2577 default newline convention (a single linefeed character) is in force: 2578 <pre> 2579 abc #comment \n still comment 2580 </pre> 2581 On encountering the # character, <b>pcre2_compile()</b> skips along, looking for 2582 a newline in the pattern. The sequence \n is still literal at this stage, so 2583 it does not terminate the comment. Only an actual character with the code value 2584 0x0a (the default newline) does so. 2585 <a name="recursion"></a></P> 2586 <br><a name="SEC23" href="#TOC1">RECURSIVE PATTERNS</a><br> 2587 <P> 2588 Consider the problem of matching a string in parentheses, allowing for 2589 unlimited nested parentheses. Without the use of recursion, the best that can 2590 be done is to use a pattern that matches up to some fixed depth of nesting. It 2591 is not possible to handle an arbitrary nesting depth. 2592 </P> 2593 <P> 2594 For some time, Perl has provided a facility that allows regular expressions to 2595 recurse (amongst other things). It does this by interpolating Perl code in the 2596 expression at run time, and the code can refer to the expression itself. A Perl 2597 pattern using code interpolation to solve the parentheses problem can be 2598 created like this: 2599 <pre> 2600 $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; 2601 </pre> 2602 The (?p{...}) item interpolates Perl code at run time, and in this case refers 2603 recursively to the pattern in which it appears. 2604 </P> 2605 <P> 2606 Obviously, PCRE2 cannot support the interpolation of Perl code. Instead, it 2607 supports special syntax for recursion of the entire pattern, and also for 2608 individual subpattern recursion. After its introduction in PCRE1 and Python, 2609 this kind of recursion was subsequently introduced into Perl at release 5.10. 2610 </P> 2611 <P> 2612 A special item that consists of (? followed by a number greater than zero and a 2613 closing parenthesis is a recursive subroutine call of the subpattern of the 2614 given number, provided that it occurs inside that subpattern. (If not, it is a 2615 <a href="#subpatternsassubroutines">non-recursive subroutine</a> 2616 call, which is described in the next section.) The special item (?R) or (?0) is 2617 a recursive call of the entire regular expression. 2618 </P> 2619 <P> 2620 This PCRE2 pattern solves the nested parentheses problem (assume the 2621 PCRE2_EXTENDED option is set so that white space is ignored): 2622 <pre> 2623 \( ( [^()]++ | (?R) )* \) 2624 </pre> 2625 First it matches an opening parenthesis. Then it matches any number of 2626 substrings which can either be a sequence of non-parentheses, or a recursive 2627 match of the pattern itself (that is, a correctly parenthesized substring). 2628 Finally there is a closing parenthesis. Note the use of a possessive quantifier 2629 to avoid backtracking into sequences of non-parentheses. 2630 </P> 2631 <P> 2632 If this were part of a larger pattern, you would not want to recurse the entire 2633 pattern, so instead you could use this: 2634 <pre> 2635 ( \( ( [^()]++ | (?1) )* \) ) 2636 </pre> 2637 We have put the pattern into parentheses, and caused the recursion to refer to 2638 them instead of the whole pattern. 2639 </P> 2640 <P> 2641 In a larger pattern, keeping track of parenthesis numbers can be tricky. This 2642 is made easier by the use of relative references. Instead of (?1) in the 2643 pattern above you can write (?-2) to refer to the second most recently opened 2644 parentheses preceding the recursion. In other words, a negative number counts 2645 capturing parentheses leftwards from the point at which it is encountered. 2646 </P> 2647 <P> 2648 Be aware however, that if 2649 <a href="#dupsubpatternnumber">duplicate subpattern numbers</a> 2650 are in use, relative references refer to the earliest subpattern with the 2651 appropriate number. Consider, for example: 2652 <pre> 2653 (?|(a)|(b)) (c) (?-2) 2654 </pre> 2655 The first two capturing groups (a) and (b) are both numbered 1, and group (c) 2656 is number 2. When the reference (?-2) is encountered, the second most recently 2657 opened parentheses has the number 1, but it is the first such group (the (a) 2658 group) to which the recursion refers. This would be the same if an absolute 2659 reference (?1) was used. In other words, relative references are just a 2660 shorthand for computing a group number. 2661 </P> 2662 <P> 2663 It is also possible to refer to subsequently opened parentheses, by writing 2664 references such as (?+2). However, these cannot be recursive because the 2665 reference is not inside the parentheses that are referenced. They are always 2666 <a href="#subpatternsassubroutines">non-recursive subroutine</a> 2667 calls, as described in the next section. 2668 </P> 2669 <P> 2670 An alternative approach is to use named parentheses. The Perl syntax for this 2671 is (?&name); PCRE1's earlier syntax (?P>name) is also supported. We could 2672 rewrite the above example as follows: 2673 <pre> 2674 (?<pn> \( ( [^()]++ | (?&pn) )* \) ) 2675 </pre> 2676 If there is more than one subpattern with the same name, the earliest one is 2677 used. 2678 </P> 2679 <P> 2680 The example pattern that we have been looking at contains nested unlimited 2681 repeats, and so the use of a possessive quantifier for matching strings of 2682 non-parentheses is important when applying the pattern to strings that do not 2683 match. For example, when this pattern is applied to 2684 <pre> 2685 (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() 2686 </pre> 2687 it yields "no match" quickly. However, if a possessive quantifier is not used, 2688 the match runs for a very long time indeed because there are so many different 2689 ways the + and * repeats can carve up the subject, and all have to be tested 2690 before failure can be reported. 2691 </P> 2692 <P> 2693 At the end of a match, the values of capturing parentheses are those from 2694 the outermost level. If you want to obtain intermediate values, a callout 2695 function can be used (see below and the 2696 <a href="pcre2callout.html"><b>pcre2callout</b></a> 2697 documentation). If the pattern above is matched against 2698 <pre> 2699 (ab(cd)ef) 2700 </pre> 2701 the value for the inner capturing parentheses (numbered 2) is "ef", which is 2702 the last value taken on at the top level. If a capturing subpattern is not 2703 matched at the top level, its final captured value is unset, even if it was 2704 (temporarily) set at a deeper level during the matching process. 2705 </P> 2706 <P> 2707 If there are more than 15 capturing parentheses in a pattern, PCRE2 has to 2708 obtain extra memory from the heap to store data during a recursion. If no 2709 memory can be obtained, the match fails with the PCRE2_ERROR_NOMEMORY error. 2710 </P> 2711 <P> 2712 Do not confuse the (?R) item with the condition (R), which tests for recursion. 2713 Consider this pattern, which matches text in angle brackets, allowing for 2714 arbitrary nesting. Only digits are allowed in nested brackets (that is, when 2715 recursing), whereas any characters are permitted at the outer level. 2716 <pre> 2717 < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > 2718 </pre> 2719 In this pattern, (?(R) is the start of a conditional subpattern, with two 2720 different alternatives for the recursive and non-recursive cases. The (?R) item 2721 is the actual recursive call. 2722 <a name="recursiondifference"></a></P> 2723 <br><b> 2724 Differences in recursion processing between PCRE2 and Perl 2725 </b><br> 2726 <P> 2727 Recursion processing in PCRE2 differs from Perl in two important ways. In PCRE2 2728 (like Python, but unlike Perl), a recursive subpattern call is always treated 2729 as an atomic group. That is, once it has matched some of the subject string, it 2730 is never re-entered, even if it contains untried alternatives and there is a 2731 subsequent matching failure. This can be illustrated by the following pattern, 2732 which purports to match a palindromic string that contains an odd number of 2733 characters (for example, "a", "aba", "abcba", "abcdcba"): 2734 <pre> 2735 ^(.|(.)(?1)\2)$ 2736 </pre> 2737 The idea is that it either matches a single character, or two identical 2738 characters surrounding a sub-palindrome. In Perl, this pattern works; in PCRE2 2739 it does not if the pattern is longer than three characters. Consider the 2740 subject string "abcba": 2741 </P> 2742 <P> 2743 At the top level, the first character is matched, but as it is not at the end 2744 of the string, the first alternative fails; the second alternative is taken 2745 and the recursion kicks in. The recursive call to subpattern 1 successfully 2746 matches the next character ("b"). (Note that the beginning and end of line 2747 tests are not part of the recursion). 2748 </P> 2749 <P> 2750 Back at the top level, the next character ("c") is compared with what 2751 subpattern 2 matched, which was "a". This fails. Because the recursion is 2752 treated as an atomic group, there are now no backtracking points, and so the 2753 entire match fails. (Perl is able, at this point, to re-enter the recursion and 2754 try the second alternative.) However, if the pattern is written with the 2755 alternatives in the other order, things are different: 2756 <pre> 2757 ^((.)(?1)\2|.)$ 2758 </pre> 2759 This time, the recursing alternative is tried first, and continues to recurse 2760 until it runs out of characters, at which point the recursion fails. But this 2761 time we do have another alternative to try at the higher level. That is the big 2762 difference: in the previous case the remaining alternative is at a deeper 2763 recursion level, which PCRE2 cannot use. 2764 </P> 2765 <P> 2766 To change the pattern so that it matches all palindromic strings, not just 2767 those with an odd number of characters, it is tempting to change the pattern to 2768 this: 2769 <pre> 2770 ^((.)(?1)\2|.?)$ 2771 </pre> 2772 Again, this works in Perl, but not in PCRE2, and for the same reason. When a 2773 deeper recursion has matched a single character, it cannot be entered again in 2774 order to match an empty string. The solution is to separate the two cases, and 2775 write out the odd and even cases as alternatives at the higher level: 2776 <pre> 2777 ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) 2778 </pre> 2779 If you want to match typical palindromic phrases, the pattern has to ignore all 2780 non-word characters, which can be done like this: 2781 <pre> 2782 ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ 2783 </pre> 2784 If run with the PCRE2_CASELESS option, this pattern matches phrases such as "A 2785 man, a plan, a canal: Panama!" and it works in both PCRE2 and Perl. Note the 2786 use of the possessive quantifier *+ to avoid backtracking into sequences of 2787 non-word characters. Without this, PCRE2 takes a great deal longer (ten times 2788 or more) to match typical phrases, and Perl takes so long that you think it has 2789 gone into a loop. 2790 </P> 2791 <P> 2792 <b>WARNING</b>: The palindrome-matching patterns above work only if the subject 2793 string does not start with a palindrome that is shorter than the entire string. 2794 For example, although "abcba" is correctly matched, if the subject is "ababa", 2795 PCRE2 finds the palindrome "aba" at the start, then fails at top level because 2796 the end of the string does not follow. Once again, it cannot jump back into the 2797 recursion to try other alternatives, so the entire match fails. 2798 </P> 2799 <P> 2800 The second way in which PCRE2 and Perl differ in their recursion processing is 2801 in the handling of captured values. In Perl, when a subpattern is called 2802 recursively or as a subpattern (see the next section), it has no access to any 2803 values that were captured outside the recursion, whereas in PCRE2 these values 2804 can be referenced. Consider this pattern: 2805 <pre> 2806 ^(.)(\1|a(?2)) 2807 </pre> 2808 In PCRE2, this pattern matches "bab". The first capturing parentheses match "b", 2809 then in the second group, when the back reference \1 fails to match "b", the 2810 second alternative matches "a" and then recurses. In the recursion, \1 does 2811 now match "b" and so the whole match succeeds. In Perl, the pattern fails to 2812 match because inside the recursive call \1 cannot access the externally set 2813 value. 2814 <a name="subpatternsassubroutines"></a></P> 2815 <br><a name="SEC24" href="#TOC1">SUBPATTERNS AS SUBROUTINES</a><br> 2816 <P> 2817 If the syntax for a recursive subpattern call (either by number or by 2818 name) is used outside the parentheses to which it refers, it operates like a 2819 subroutine in a programming language. The called subpattern may be defined 2820 before or after the reference. A numbered reference can be absolute or 2821 relative, as in these examples: 2822 <pre> 2823 (...(absolute)...)...(?2)... 2824 (...(relative)...)...(?-1)... 2825 (...(?+1)...(relative)... 2826 </pre> 2827 An earlier example pointed out that the pattern 2828 <pre> 2829 (sens|respons)e and \1ibility 2830 </pre> 2831 matches "sense and sensibility" and "response and responsibility", but not 2832 "sense and responsibility". If instead the pattern 2833 <pre> 2834 (sens|respons)e and (?1)ibility 2835 </pre> 2836 is used, it does match "sense and responsibility" as well as the other two 2837 strings. Another example is given in the discussion of DEFINE above. 2838 </P> 2839 <P> 2840 All subroutine calls, whether recursive or not, are always treated as atomic 2841 groups. That is, once a subroutine has matched some of the subject string, it 2842 is never re-entered, even if it contains untried alternatives and there is a 2843 subsequent matching failure. Any capturing parentheses that are set during the 2844 subroutine call revert to their previous values afterwards. 2845 </P> 2846 <P> 2847 Processing options such as case-independence are fixed when a subpattern is 2848 defined, so if it is used as a subroutine, such options cannot be changed for 2849 different calls. For example, consider this pattern: 2850 <pre> 2851 (abc)(?i:(?-1)) 2852 </pre> 2853 It matches "abcabc". It does not match "abcABC" because the change of 2854 processing option does not affect the called subpattern. 2855 <a name="onigurumasubroutines"></a></P> 2856 <br><a name="SEC25" href="#TOC1">ONIGURUMA SUBROUTINE SYNTAX</a><br> 2857 <P> 2858 For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or 2859 a number enclosed either in angle brackets or single quotes, is an alternative 2860 syntax for referencing a subpattern as a subroutine, possibly recursively. Here 2861 are two of the examples used above, rewritten using this syntax: 2862 <pre> 2863 (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) 2864 (sens|respons)e and \g'1'ibility 2865 </pre> 2866 PCRE2 supports an extension to Oniguruma: if a number is preceded by a 2867 plus or a minus sign it is taken as a relative reference. For example: 2868 <pre> 2869 (abc)(?i:\g<-1>) 2870 </pre> 2871 Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> 2872 synonymous. The former is a back reference; the latter is a subroutine call. 2873 </P> 2874 <br><a name="SEC26" href="#TOC1">CALLOUTS</a><br> 2875 <P> 2876 Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl 2877 code to be obeyed in the middle of matching a regular expression. This makes it 2878 possible, amongst other things, to extract different substrings that match the 2879 same pair of parentheses when there is a repetition. 2880 </P> 2881 <P> 2882 PCRE2 provides a similar feature, but of course it cannot obey arbitrary Perl 2883 code. The feature is called "callout". The caller of PCRE2 provides an external 2884 function by putting its entry point in a match context using the function 2885 <b>pcre2_set_callout()</b>, and then passing that context to <b>pcre2_match()</b> 2886 or <b>pcre2_dfa_match()</b>. If no match context is passed, or if the callout 2887 entry point is set to NULL, callouts are disabled. 2888 </P> 2889 <P> 2890 Within a regular expression, (?C<arg>) indicates a point at which the external 2891 function is to be called. There are two kinds of callout: those with a 2892 numerical argument and those with a string argument. (?C) on its own with no 2893 argument is treated as (?C0). A numerical argument allows the application to 2894 distinguish between different callouts. String arguments were added for release 2895 10.20 to make it possible for script languages that use PCRE2 to embed short 2896 scripts within patterns in a similar way to Perl. 2897 </P> 2898 <P> 2899 During matching, when PCRE2 reaches a callout point, the external function is 2900 called. It is provided with the number or string argument of the callout, the 2901 position in the pattern, and one item of data that is also set in the match 2902 block. The callout function may cause matching to proceed, to backtrack, or to 2903 fail. 2904 </P> 2905 <P> 2906 By default, PCRE2 implements a number of optimizations at matching time, and 2907 one side-effect is that sometimes callouts are skipped. If you need all 2908 possible callouts to happen, you need to set options that disable the relevant 2909 optimizations. More details, including a complete description of the 2910 programming interface to the callout function, are given in the 2911 <a href="pcre2callout.html"><b>pcre2callout</b></a> 2912 documentation. 2913 </P> 2914 <br><b> 2915 Callouts with numerical arguments 2916 </b><br> 2917 <P> 2918 If you just want to have a means of identifying different callout points, put a 2919 number less than 256 after the letter C. For example, this pattern has two 2920 callout points: 2921 <pre> 2922 (?C1)abc(?C2)def 2923 </pre> 2924 If the PCRE2_AUTO_CALLOUT flag is passed to <b>pcre2_compile()</b>, numerical 2925 callouts are automatically installed before each item in the pattern. They are 2926 all numbered 255. If there is a conditional group in the pattern whose 2927 condition is an assertion, an additional callout is inserted just before the 2928 condition. An explicit callout may also be set at this position, as in this 2929 example: 2930 <pre> 2931 (?(?C9)(?=a)abc|def) 2932 </pre> 2933 Note that this applies only to assertion conditions, not to other types of 2934 condition. 2935 </P> 2936 <br><b> 2937 Callouts with string arguments 2938 </b><br> 2939 <P> 2940 A delimited string may be used instead of a number as a callout argument. The 2941 starting delimiter must be one of ` ' " ^ % # $ { and the ending delimiter is 2942 the same as the start, except for {, where the ending delimiter is }. If the 2943 ending delimiter is needed within the string, it must be doubled. For 2944 example: 2945 <pre> 2946 (?C'ab ''c'' d')xyz(?C{any text})pqr 2947 </pre> 2948 The doubling is removed before the string is passed to the callout function. 2949 <a name="backtrackcontrol"></a></P> 2950 <br><a name="SEC27" href="#TOC1">BACKTRACKING CONTROL</a><br> 2951 <P> 2952 Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which 2953 are still described in the Perl documentation as "experimental and subject to 2954 change or removal in a future version of Perl". It goes on to say: "Their usage 2955 in production code should be noted to avoid problems during upgrades." The same 2956 remarks apply to the PCRE2 features described in this section. 2957 </P> 2958 <P> 2959 The new verbs make use of what was previously invalid syntax: an opening 2960 parenthesis followed by an asterisk. They are generally of the form (*VERB) or 2961 (*VERB:NAME). Some verbs take either form, possibly behaving differently 2962 depending on whether or not a name is present. 2963 </P> 2964 <P> 2965 By default, for compatibility with Perl, a name is any sequence of characters 2966 that does not include a closing parenthesis. The name is not processed in 2967 any way, and it is not possible to include a closing parenthesis in the name. 2968 However, if the PCRE2_ALT_VERBNAMES option is set, normal backslash processing 2969 is applied to verb names and only an unescaped closing parenthesis terminates 2970 the name. A closing parenthesis can be included in a name either as \) or 2971 between \Q and \E. If the PCRE2_EXTENDED option is set, unescaped whitespace 2972 in verb names is skipped and #-comments are recognized, exactly as in the rest 2973 of the pattern. 2974 </P> 2975 <P> 2976 The maximum length of a name is 255 in the 8-bit library and 65535 in the 2977 16-bit and 32-bit libraries. If the name is empty, that is, if the closing 2978 parenthesis immediately follows the colon, the effect is as if the colon were 2979 not there. Any number of these verbs may occur in a pattern. 2980 </P> 2981 <P> 2982 Since these verbs are specifically related to backtracking, most of them can be 2983 used only when the pattern is to be matched using the traditional matching 2984 function, because these use a backtracking algorithm. With the exception of 2985 (*FAIL), which behaves like a failing negative assertion, the backtracking 2986 control verbs cause an error if encountered by the DFA matching function. 2987 </P> 2988 <P> 2989 The behaviour of these verbs in 2990 <a href="#btrepeat">repeated groups,</a> 2991 <a href="#btassert">assertions,</a> 2992 and in 2993 <a href="#btsub">subpatterns called as subroutines</a> 2994 (whether or not recursively) is documented below. 2995 <a name="nooptimize"></a></P> 2996 <br><b> 2997 Optimizations that affect backtracking verbs 2998 </b><br> 2999 <P> 3000 PCRE2 contains some optimizations that are used to speed up matching by running 3001 some checks at the start of each match attempt. For example, it may know the 3002 minimum length of matching subject, or that a particular character must be 3003 present. When one of these optimizations bypasses the running of a match, any 3004 included backtracking verbs will not, of course, be processed. You can suppress 3005 the start-of-match optimizations by setting the PCRE2_NO_START_OPTIMIZE option 3006 when calling <b>pcre2_compile()</b>, or by starting the pattern with 3007 (*NO_START_OPT). There is more discussion of this option in the section 3008 entitled 3009 <a href="pcre2api.html#compiling">"Compiling a pattern"</a> 3010 in the 3011 <a href="pcre2api.html"><b>pcre2api</b></a> 3012 documentation. 3013 </P> 3014 <P> 3015 Experiments with Perl suggest that it too has similar optimizations, sometimes 3016 leading to anomalous results. 3017 </P> 3018 <br><b> 3019 Verbs that act immediately 3020 </b><br> 3021 <P> 3022 The following verbs act as soon as they are encountered. They may not be 3023 followed by a name. 3024 <pre> 3025 (*ACCEPT) 3026 </pre> 3027 This verb causes the match to end successfully, skipping the remainder of the 3028 pattern. However, when it is inside a subpattern that is called as a 3029 subroutine, only that subpattern is ended successfully. Matching then continues 3030 at the outer level. If (*ACCEPT) in triggered in a positive assertion, the 3031 assertion succeeds; in a negative assertion, the assertion fails. 3032 </P> 3033 <P> 3034 If (*ACCEPT) is inside capturing parentheses, the data so far is captured. For 3035 example: 3036 <pre> 3037 A((?:A|B(*ACCEPT)|C)D) 3038 </pre> 3039 This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is captured by 3040 the outer parentheses. 3041 <pre> 3042 (*FAIL) or (*F) 3043 </pre> 3044 This verb causes a matching failure, forcing backtracking to occur. It is 3045 equivalent to (?!) but easier to read. The Perl documentation notes that it is 3046 probably useful only when combined with (?{}) or (??{}). Those are, of course, 3047 Perl features that are not present in PCRE2. The nearest equivalent is the 3048 callout feature, as for example in this pattern: 3049 <pre> 3050 a+(?C)(*FAIL) 3051 </pre> 3052 A match with the string "aaaa" always fails, but the callout is taken before 3053 each backtrack happens (in this example, 10 times). 3054 </P> 3055 <br><b> 3056 Recording which path was taken 3057 </b><br> 3058 <P> 3059 There is one verb whose main purpose is to track how a match was arrived at, 3060 though it also has a secondary use in conjunction with advancing the match 3061 starting point (see (*SKIP) below). 3062 <pre> 3063 (*MARK:NAME) or (*:NAME) 3064 </pre> 3065 A name is always required with this verb. There may be as many instances of 3066 (*MARK) as you like in a pattern, and their names do not have to be unique. 3067 </P> 3068 <P> 3069 When a match succeeds, the name of the last-encountered (*MARK:NAME), 3070 (*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to the 3071 caller as described in the section entitled 3072 <a href="pcre2api.html#matchotherdata">"Other information about the match"</a> 3073 in the 3074 <a href="pcre2api.html"><b>pcre2api</b></a> 3075 documentation. Here is an example of <b>pcre2test</b> output, where the "mark" 3076 modifier requests the retrieval and outputting of (*MARK) data: 3077 <pre> 3078 re> /X(*MARK:A)Y|X(*MARK:B)Z/mark 3079 data> XY 3080 0: XY 3081 MK: A 3082 XZ 3083 0: XZ 3084 MK: B 3085 </pre> 3086 The (*MARK) name is tagged with "MK:" in this output, and in this example it 3087 indicates which of the two alternatives matched. This is a more efficient way 3088 of obtaining this information than putting each alternative in its own 3089 capturing parentheses. 3090 </P> 3091 <P> 3092 If a verb with a name is encountered in a positive assertion that is true, the 3093 name is recorded and passed back if it is the last-encountered. This does not 3094 happen for negative assertions or failing positive assertions. 3095 </P> 3096 <P> 3097 After a partial match or a failed match, the last encountered name in the 3098 entire match process is returned. For example: 3099 <pre> 3100 re> /X(*MARK:A)Y|X(*MARK:B)Z/mark 3101 data> XP 3102 No match, mark = B 3103 </pre> 3104 Note that in this unanchored example the mark is retained from the match 3105 attempt that started at the letter "X" in the subject. Subsequent match 3106 attempts starting at "P" and then with an empty string do not get as far as the 3107 (*MARK) item, but nevertheless do not reset it. 3108 </P> 3109 <P> 3110 If you are interested in (*MARK) values after failed matches, you should 3111 probably set the PCRE2_NO_START_OPTIMIZE option 3112 <a href="#nooptimize">(see above)</a> 3113 to ensure that the match is always attempted. 3114 </P> 3115 <br><b> 3116 Verbs that act after backtracking 3117 </b><br> 3118 <P> 3119 The following verbs do nothing when they are encountered. Matching continues 3120 with what follows, but if there is no subsequent match, causing a backtrack to 3121 the verb, a failure is forced. That is, backtracking cannot pass to the left of 3122 the verb. However, when one of these verbs appears inside an atomic group 3123 (which includes any group that is called as a subroutine) or in an assertion 3124 that is true, its effect is confined to that group, because once the group has 3125 been matched, there is never any backtracking into it. In this situation, 3126 backtracking has to jump to the left of the entire atomic group or assertion. 3127 </P> 3128 <P> 3129 These verbs differ in exactly what kind of failure occurs when backtracking 3130 reaches them. The behaviour described below is what happens when the verb is 3131 not in a subroutine or an assertion. Subsequent sections cover these special 3132 cases. 3133 <pre> 3134 (*COMMIT) 3135 </pre> 3136 This verb, which may not be followed by a name, causes the whole match to fail 3137 outright if there is a later matching failure that causes backtracking to reach 3138 it. Even if the pattern is unanchored, no further attempts to find a match by 3139 advancing the starting point take place. If (*COMMIT) is the only backtracking 3140 verb that is encountered, once it has been passed <b>pcre2_match()</b> is 3141 committed to finding a match at the current starting point, or not at all. For 3142 example: 3143 <pre> 3144 a+(*COMMIT)b 3145 </pre> 3146 This matches "xxaab" but not "aacaab". It can be thought of as a kind of 3147 dynamic anchor, or "I've started, so I must finish." The name of the most 3148 recently passed (*MARK) in the path is passed back when (*COMMIT) forces a 3149 match failure. 3150 </P> 3151 <P> 3152 If there is more than one backtracking verb in a pattern, a different one that 3153 follows (*COMMIT) may be triggered first, so merely passing (*COMMIT) during a 3154 match does not always guarantee that a match must be at this starting point. 3155 </P> 3156 <P> 3157 Note that (*COMMIT) at the start of a pattern is not the same as an anchor, 3158 unless PCRE2's start-of-match optimizations are turned off, as shown in this 3159 output from <b>pcre2test</b>: 3160 <pre> 3161 re> /(*COMMIT)abc/ 3162 data> xyzabc 3163 0: abc 3164 data> 3165 re> /(*COMMIT)abc/no_start_optimize 3166 data> xyzabc 3167 No match 3168 </pre> 3169 For the first pattern, PCRE2 knows that any match must start with "a", so the 3170 optimization skips along the subject to "a" before applying the pattern to the 3171 first set of data. The match attempt then succeeds. The second pattern disables 3172 the optimization that skips along to the first character. The pattern is now 3173 applied starting at "x", and so the (*COMMIT) causes the match to fail without 3174 trying any other starting points. 3175 <pre> 3176 (*PRUNE) or (*PRUNE:NAME) 3177 </pre> 3178 This verb causes the match to fail at the current starting position in the 3179 subject if there is a later matching failure that causes backtracking to reach 3180 it. If the pattern is unanchored, the normal "bumpalong" advance to the next 3181 starting character then happens. Backtracking can occur as usual to the left of 3182 (*PRUNE), before it is reached, or when matching to the right of (*PRUNE), but 3183 if there is no match to the right, backtracking cannot cross (*PRUNE). In 3184 simple cases, the use of (*PRUNE) is just an alternative to an atomic group or 3185 possessive quantifier, but there are some uses of (*PRUNE) that cannot be 3186 expressed in any other way. In an anchored pattern (*PRUNE) has the same effect 3187 as (*COMMIT). 3188 </P> 3189 <P> 3190 The behaviour of (*PRUNE:NAME) is the not the same as (*MARK:NAME)(*PRUNE). 3191 It is like (*MARK:NAME) in that the name is remembered for passing back to the 3192 caller. However, (*SKIP:NAME) searches only for names set with (*MARK), 3193 ignoring those set by (*PRUNE) or (*THEN). 3194 <pre> 3195 (*SKIP) 3196 </pre> 3197 This verb, when given without a name, is like (*PRUNE), except that if the 3198 pattern is unanchored, the "bumpalong" advance is not to the next character, 3199 but to the position in the subject where (*SKIP) was encountered. (*SKIP) 3200 signifies that whatever text was matched leading up to it cannot be part of a 3201 successful match. Consider: 3202 <pre> 3203 a+(*SKIP)b 3204 </pre> 3205 If the subject is "aaaac...", after the first match attempt fails (starting at 3206 the first character in the string), the starting point skips on to start the 3207 next attempt at "c". Note that a possessive quantifer does not have the same 3208 effect as this example; although it would suppress backtracking during the 3209 first match attempt, the second attempt would start at the second character 3210 instead of skipping on to "c". 3211 <pre> 3212 (*SKIP:NAME) 3213 </pre> 3214 When (*SKIP) has an associated name, its behaviour is modified. When it is 3215 triggered, the previous path through the pattern is searched for the most 3216 recent (*MARK) that has the same name. If one is found, the "bumpalong" advance 3217 is to the subject position that corresponds to that (*MARK) instead of to where 3218 (*SKIP) was encountered. If no (*MARK) with a matching name is found, the 3219 (*SKIP) is ignored. 3220 </P> 3221 <P> 3222 Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores 3223 names that are set by (*PRUNE:NAME) or (*THEN:NAME). 3224 <pre> 3225 (*THEN) or (*THEN:NAME) 3226 </pre> 3227 This verb causes a skip to the next innermost alternative when backtracking 3228 reaches it. That is, it cancels any further backtracking within the current 3229 alternative. Its name comes from the observation that it can be used for a 3230 pattern-based if-then-else block: 3231 <pre> 3232 ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... 3233 </pre> 3234 If the COND1 pattern matches, FOO is tried (and possibly further items after 3235 the end of the group if FOO succeeds); on failure, the matcher skips to the 3236 second alternative and tries COND2, without backtracking into COND1. If that 3237 succeeds and BAR fails, COND3 is tried. If subsequently BAZ fails, there are no 3238 more alternatives, so there is a backtrack to whatever came before the entire 3239 group. If (*THEN) is not inside an alternation, it acts like (*PRUNE). 3240 </P> 3241 <P> 3242 The behaviour of (*THEN:NAME) is the not the same as (*MARK:NAME)(*THEN). 3243 It is like (*MARK:NAME) in that the name is remembered for passing back to the 3244 caller. However, (*SKIP:NAME) searches only for names set with (*MARK), 3245 ignoring those set by (*PRUNE) and (*THEN). 3246 </P> 3247 <P> 3248 A subpattern that does not contain a | character is just a part of the 3249 enclosing alternative; it is not a nested alternation with only one 3250 alternative. The effect of (*THEN) extends beyond such a subpattern to the 3251 enclosing alternative. Consider this pattern, where A, B, etc. are complex 3252 pattern fragments that do not contain any | characters at this level: 3253 <pre> 3254 A (B(*THEN)C) | D 3255 </pre> 3256 If A and B are matched, but there is a failure in C, matching does not 3257 backtrack into A; instead it moves to the next alternative, that is, D. 3258 However, if the subpattern containing (*THEN) is given an alternative, it 3259 behaves differently: 3260 <pre> 3261 A (B(*THEN)C | (*FAIL)) | D 3262 </pre> 3263 The effect of (*THEN) is now confined to the inner subpattern. After a failure 3264 in C, matching moves to (*FAIL), which causes the whole subpattern to fail 3265 because there are no more alternatives to try. In this case, matching does now 3266 backtrack into A. 3267 </P> 3268 <P> 3269 Note that a conditional subpattern is not considered as having two 3270 alternatives, because only one is ever used. In other words, the | character in 3271 a conditional subpattern has a different meaning. Ignoring white space, 3272 consider: 3273 <pre> 3274 ^.*? (?(?=a) a | b(*THEN)c ) 3275 </pre> 3276 If the subject is "ba", this pattern does not match. Because .*? is ungreedy, 3277 it initially matches zero characters. The condition (?=a) then fails, the 3278 character "b" is matched, but "c" is not. At this point, matching does not 3279 backtrack to .*? as might perhaps be expected from the presence of the | 3280 character. The conditional subpattern is part of the single alternative that 3281 comprises the whole pattern, and so the match fails. (If there was a backtrack 3282 into .*?, allowing it to match "b", the match would succeed.) 3283 </P> 3284 <P> 3285 The verbs just described provide four different "strengths" of control when 3286 subsequent matching fails. (*THEN) is the weakest, carrying on the match at the 3287 next alternative. (*PRUNE) comes next, failing the match at the current 3288 starting position, but allowing an advance to the next character (for an 3289 unanchored pattern). (*SKIP) is similar, except that the advance may be more 3290 than one character. (*COMMIT) is the strongest, causing the entire match to 3291 fail. 3292 </P> 3293 <br><b> 3294 More than one backtracking verb 3295 </b><br> 3296 <P> 3297 If more than one backtracking verb is present in a pattern, the one that is 3298 backtracked onto first acts. For example, consider this pattern, where A, B, 3299 etc. are complex pattern fragments: 3300 <pre> 3301 (A(*COMMIT)B(*THEN)C|ABD) 3302 </pre> 3303 If A matches but B fails, the backtrack to (*COMMIT) causes the entire match to 3304 fail. However, if A and B match, but C fails, the backtrack to (*THEN) causes 3305 the next alternative (ABD) to be tried. This behaviour is consistent, but is 3306 not always the same as Perl's. It means that if two or more backtracking verbs 3307 appear in succession, all the the last of them has no effect. Consider this 3308 example: 3309 <pre> 3310 ...(*COMMIT)(*PRUNE)... 3311 </pre> 3312 If there is a matching failure to the right, backtracking onto (*PRUNE) causes 3313 it to be triggered, and its action is taken. There can never be a backtrack 3314 onto (*COMMIT). 3315 <a name="btrepeat"></a></P> 3316 <br><b> 3317 Backtracking verbs in repeated groups 3318 </b><br> 3319 <P> 3320 PCRE2 differs from Perl in its handling of backtracking verbs in repeated 3321 groups. For example, consider: 3322 <pre> 3323 /(a(*COMMIT)b)+ac/ 3324 </pre> 3325 If the subject is "abac", Perl matches, but PCRE2 fails because the (*COMMIT) 3326 in the second repeat of the group acts. 3327 <a name="btassert"></a></P> 3328 <br><b> 3329 Backtracking verbs in assertions 3330 </b><br> 3331 <P> 3332 (*FAIL) in an assertion has its normal effect: it forces an immediate 3333 backtrack. 3334 </P> 3335 <P> 3336 (*ACCEPT) in a positive assertion causes the assertion to succeed without any 3337 further processing. In a negative assertion, (*ACCEPT) causes the assertion to 3338 fail without any further processing. 3339 </P> 3340 <P> 3341 The other backtracking verbs are not treated specially if they appear in a 3342 positive assertion. In particular, (*THEN) skips to the next alternative in the 3343 innermost enclosing group that has alternations, whether or not this is within 3344 the assertion. 3345 </P> 3346 <P> 3347 Negative assertions are, however, different, in order to ensure that changing a 3348 positive assertion into a negative assertion changes its result. Backtracking 3349 into (*COMMIT), (*SKIP), or (*PRUNE) causes a negative assertion to be true, 3350 without considering any further alternative branches in the assertion. 3351 Backtracking into (*THEN) causes it to skip to the next enclosing alternative 3352 within the assertion (the normal behaviour), but if the assertion does not have 3353 such an alternative, (*THEN) behaves like (*PRUNE). 3354 <a name="btsub"></a></P> 3355 <br><b> 3356 Backtracking verbs in subroutines 3357 </b><br> 3358 <P> 3359 These behaviours occur whether or not the subpattern is called recursively. 3360 Perl's treatment of subroutines is different in some cases. 3361 </P> 3362 <P> 3363 (*FAIL) in a subpattern called as a subroutine has its normal effect: it forces 3364 an immediate backtrack. 3365 </P> 3366 <P> 3367 (*ACCEPT) in a subpattern called as a subroutine causes the subroutine match to 3368 succeed without any further processing. Matching then continues after the 3369 subroutine call. 3370 </P> 3371 <P> 3372 (*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine cause 3373 the subroutine match to fail. 3374 </P> 3375 <P> 3376 (*THEN) skips to the next alternative in the innermost enclosing group within 3377 the subpattern that has alternatives. If there is no such group within the 3378 subpattern, (*THEN) causes the subroutine match to fail. 3379 </P> 3380 <br><a name="SEC28" href="#TOC1">SEE ALSO</a><br> 3381 <P> 3382 <b>pcre2api</b>(3), <b>pcre2callout</b>(3), <b>pcre2matching</b>(3), 3383 <b>pcre2syntax</b>(3), <b>pcre2</b>(3). 3384 </P> 3385 <br><a name="SEC29" href="#TOC1">AUTHOR</a><br> 3386 <P> 3387 Philip Hazel 3388 <br> 3389 University Computing Service 3390 <br> 3391 Cambridge, England. 3392 <br> 3393 </P> 3394 <br><a name="SEC30" href="#TOC1">REVISION</a><br> 3395 <P> 3396 Last updated: 20 June 2016 3397 <br> 3398 Copyright © 1997-2016 University of Cambridge. 3399 <br> 3400 <p> 3401 Return to the <a href="index.html">PCRE2 index page</a>. 3402 </p> 3403
