1 <refentry id="glib-regex-syntax" revision="11 Jul 2006"> 2 <refmeta> 3 <refentrytitle>Regular expression syntax</refentrytitle> 4 </refmeta> 5 6 <!-- 7 Based on the man page for pcrepattern. 8 9 Remember to sync this document with the file docs/pcrepattern.3 in the 10 pcre package when upgrading to a newer version of pcre. 11 12 In sync with PCRE 7.0 13 --> 14 15 <refnamediv> 16 <refname>Regular expression syntax</refname> 17 <refpurpose> 18 Syntax and semantics of the regular expressions supported by GRegex 19 </refpurpose> 20 </refnamediv> 21 22 <refsect1> 23 <title>GRegex regular expression details</title> 24 <para> 25 A regular expression is a pattern that is matched against a 26 string from left to right. Most characters stand for themselves in a 27 pattern, and match the corresponding characters in the string. As a 28 trivial example, the pattern 29 </para> 30 31 <programlisting> 32 The quick brown fox 33 </programlisting> 34 35 <para> 36 matches a portion of a string that is identical to itself. When 37 caseless matching is specified (the <varname>G_REGEX_CASELESS</varname> flag), letters are 38 matched independently of case. 39 </para> 40 41 <para> 42 The power of regular expressions comes from the ability to include 43 alternatives and repetitions in the pattern. These are encoded in the 44 pattern by the use of metacharacters, which do not stand for themselves 45 but instead are interpreted in some special way. 46 </para> 47 48 <para> 49 There are two different sets of metacharacters: those that are recognized 50 anywhere in the pattern except within square brackets, and those 51 that are recognized in square brackets. Outside square brackets, the 52 metacharacters are as follows: 53 </para> 54 55 <table frame="all" colsep="1" rowsep="1"> 56 <title>Metacharacters outside square brackets</title> 57 <tgroup cols="2"> 58 <colspec colnum="1" align="center"/> 59 <thead> 60 <row> 61 <entry>Character</entry> 62 <entry>Meaning</entry> 63 </row> 64 </thead> 65 <tbody> 66 <row> 67 <entry>\</entry> 68 <entry>general escape character with several uses</entry> 69 </row> 70 <row> 71 <entry>^</entry> 72 <entry>assert start of string (or line, in multiline mode)</entry> 73 </row> 74 <row> 75 <entry>$</entry> 76 <entry>assert end of string (or line, in multiline mode)</entry> 77 </row> 78 <row> 79 <entry>.</entry> 80 <entry>match any character except newline (by default)</entry> 81 </row> 82 <row> 83 <entry>[</entry> 84 <entry>start character class definition</entry> 85 </row> 86 <row> 87 <entry>|</entry> 88 <entry>start of alternative branch</entry> 89 </row> 90 <row> 91 <entry>(</entry> 92 <entry>start subpattern</entry> 93 </row> 94 <row> 95 <entry>)</entry> 96 <entry>end subpattern</entry> 97 </row> 98 <row> 99 <entry>?</entry> 100 <entry>extends the meaning of (, or 0/1 quantifier, or quantifier minimizer</entry> 101 </row> 102 <row> 103 <entry>*</entry> 104 <entry>0 or more quantifier</entry> 105 </row> 106 <row> 107 <entry>+</entry> 108 <entry>1 or more quantifier, also "possessive quantifier"</entry> 109 </row> 110 <row> 111 <entry>{</entry> 112 <entry>start min/max quantifier</entry> 113 </row> 114 </tbody> 115 </tgroup> 116 </table> 117 118 <para> 119 Part of a pattern that is in square brackets is called a "character 120 class". In a character class the only metacharacters are: 121 </para> 122 123 <table frame="all" colsep="1" rowsep="1"> 124 <title>Metacharacters inside square brackets</title> 125 <tgroup cols="2"> 126 <colspec colnum="1" align="center"/> 127 <thead> 128 <row> 129 <entry>Character</entry> 130 <entry>Meaning</entry> 131 </row> 132 </thead> 133 <tbody> 134 <row> 135 <entry>\</entry> 136 <entry>general escape character</entry> 137 </row> 138 <row> 139 <entry>^</entry> 140 <entry>negate the class, but only if the first character</entry> 141 </row> 142 <row> 143 <entry>-</entry> 144 <entry>indicates character range</entry> 145 </row> 146 <row> 147 <entry>[</entry> 148 <entry>POSIX character class (only if followed by POSIX syntax)</entry> 149 </row> 150 <row> 151 <entry>]</entry> 152 <entry>terminates the character class</entry> 153 </row> 154 </tbody> 155 </tgroup> 156 </table> 157 </refsect1> 158 159 <refsect1> 160 <title>Backslash</title> 161 <para> 162 The backslash character has several uses. Firstly, if it is followed by 163 a non-alphanumeric character, it takes away any special meaning that 164 character may have. This use of backslash as an escape character 165 applies both inside and outside character classes. 166 </para> 167 168 <para> 169 For example, if you want to match a * character, you write \* in the 170 pattern. This escaping action applies whether or not the following 171 character would otherwise be interpreted as a metacharacter, so it is 172 always safe to precede a non-alphanumeric with backslash to specify 173 that it stands for itself. In particular, if you want to match a 174 backslash, you write \\. 175 </para> 176 177 <para> 178 If a pattern is compiled with the <varname>G_REGEX_EXTENDED</varname> 179 option, whitespace in the pattern (other than in a character class) and 180 characters between a # outside a character class and the next newline 181 are ignored. 182 An escaping backslash can be used to include a whitespace or # character 183 as part of the pattern. 184 </para> 185 186 <para> 187 If you want to remove the special meaning from a sequence of characters, 188 you can do so by putting them between \Q and \E. 189 The \Q...\E sequence is recognized both inside and outside character 190 classes. 191 </para> 192 193 <refsect2> 194 <title>Non-printing characters</title> 195 <para> 196 A second use of backslash provides a way of encoding non-printing 197 characters in patterns in a visible manner. There is no restriction on the 198 appearance of non-printing characters, apart from the binary zero that 199 terminates a pattern, but when a pattern is being prepared by text 200 editing, it is usually easier to use one of the following escape 201 sequences than the binary character it represents: 202 </para> 203 204 <table frame="all" colsep="1" rowsep="1"> 205 <title>Non-printing characters</title> 206 <tgroup cols="2"> 207 <colspec colnum="1" align="center"/> 208 <thead> 209 <row> 210 <entry>Escape</entry> 211 <entry>Meaning</entry> 212 </row> 213 </thead> 214 <tbody> 215 <row> 216 <entry>\a</entry> 217 <entry>alarm, that is, the BEL character (hex 07)</entry> 218 </row> 219 <row> 220 <entry>\cx</entry> 221 <entry>"control-x", where x is any character</entry> 222 </row> 223 <row> 224 <entry>\e</entry> 225 <entry>escape (hex 1B)</entry> 226 </row> 227 <row> 228 <entry>\f</entry> 229 <entry>formfeed (hex 0C)</entry> 230 </row> 231 <row> 232 <entry>\n</entry> 233 <entry>newline (hex 0A)</entry> 234 </row> 235 <row> 236 <entry>\r</entry> 237 <entry>carriage return (hex 0D)</entry> 238 </row> 239 <row> 240 <entry>\t</entry> 241 <entry>tab (hex 09)</entry> 242 </row> 243 <row> 244 <entry>\ddd</entry> 245 <entry>character with octal code ddd, or backreference</entry> 246 </row> 247 <row> 248 <entry>\xhh</entry> 249 <entry>character with hex code hh</entry> 250 </row> 251 <row> 252 <entry>\x{hhh..}</entry> 253 <entry>character with hex code hhh..</entry> 254 </row> 255 </tbody> 256 </tgroup> 257 </table> 258 259 <para> 260 The precise effect of \cx is as follows: if x is a lower case letter, 261 it is converted to upper case. Then bit 6 of the character (hex 40) is 262 inverted. Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; 263 becomes hex 7B. 264 </para> 265 266 <para> 267 After \x, from zero to two hexadecimal digits are read (letters can be 268 in upper or lower case). Any number of hexadecimal digits may appear 269 between \x{ and }, but the value of the character code 270 must be less than 2**31 (that is, the maximum hexadecimal value is 271 7FFFFFFF). If characters other than hexadecimal digits appear between 272 \x{ and }, or if there is no terminating }, this form of escape is not 273 recognized. Instead, the initial \x will be interpreted as a basic hexadecimal 274 escape, with no following digits, giving a character whose 275 value is zero. 276 </para> 277 278 <para> 279 Characters whose value is less than 256 can be defined by either of the 280 two syntaxes for \x. There is no difference 281 in the way they are handled. For example, \xdc is exactly the same as 282 \x{dc}. 283 </para> 284 285 <para> 286 After \0 up to two further octal digits are read. If there are fewer 287 than two digits, just those that are present are used. 288 Thus the sequence \0\x\07 specifies two binary zeros followed by a BEL 289 character (code value 7). Make sure you supply two digits after the 290 initial zero if the pattern character that follows is itself an octal 291 digit. 292 </para> 293 294 <para> 295 The handling of a backslash followed by a digit other than 0 is complicated. 296 Outside a character class, GRegex reads it and any following digits as a 297 decimal number. If the number is less than 10, or if there 298 have been at least that many previous capturing left parentheses in the 299 expression, the entire sequence is taken as a back reference. A 300 description of how this works is given later, following the discussion 301 of parenthesized subpatterns. 302 </para> 303 304 <para> 305 Inside a character class, or if the decimal number is greater than 9 306 and there have not been that many capturing subpatterns, GRegex re-reads 307 up to three octal digits following the backslash, and uses them to generate 308 a data character. Any subsequent digits stand for themselves. For example: 309 </para> 310 311 <table frame="all" colsep="1" rowsep="1"> 312 <title>Non-printing characters</title> 313 <tgroup cols="2"> 314 <colspec colnum="1" align="center"/> 315 <thead> 316 <row> 317 <entry>Escape</entry> 318 <entry>Meaning</entry> 319 </row> 320 </thead> 321 <tbody> 322 <row> 323 <entry>\040</entry> 324 <entry>is another way of writing a space</entry> 325 </row> 326 <row> 327 <entry>\40</entry> 328 <entry>is the same, provided there are fewer than 40 previous capturing subpatterns</entry> 329 </row> 330 <row> 331 <entry>\7</entry> 332 <entry>is always a back reference</entry> 333 </row> 334 <row> 335 <entry>\11</entry> 336 <entry>might be a back reference, or another way of writing a tab</entry> 337 </row> 338 <row> 339 <entry>\011</entry> 340 <entry>is always a tab</entry> 341 </row> 342 <row> 343 <entry>\0113</entry> 344 <entry>is a tab followed by the character "3"</entry> 345 </row> 346 <row> 347 <entry>\113</entry> 348 <entry>might be a back reference, otherwise the character with octal code 113</entry> 349 </row> 350 <row> 351 <entry>\377</entry> 352 <entry>might be a back reference, otherwise the byte consisting entirely of 1 bits</entry> 353 </row> 354 <row> 355 <entry>\81</entry> 356 <entry>is either a back reference, or a binary zero followed by the two characters "8" and "1"</entry> 357 </row> 358 </tbody> 359 </tgroup> 360 </table> 361 362 <para> 363 Note that octal values of 100 or greater must not be introduced by a 364 leading zero, because no more than three octal digits are ever read. 365 </para> 366 367 <para> 368 All the sequences that define a single character can be used both inside 369 and outside character classes. In addition, inside a character class, the 370 sequence \b is interpreted as the backspace character (hex 08), and the 371 sequences \R and \X are interpreted as the characters "R" and "X", respectively. 372 Outside a character class, these sequences have different meanings (see below). 373 </para> 374 </refsect2> 375 376 <refsect2> 377 <title>Absolute and relative back references</title> 378 <para> 379 The sequence \g followed by a positive or negative number, optionally enclosed 380 in braces, is an absolute or relative back reference. Back references are 381 discussed later, following the discussion of parenthesized subpatterns. 382 </para> 383 </refsect2> 384 385 <refsect2> 386 <title>Generic character types</title> 387 388 <para> 389 Another use of backslash is for specifying generic character types. 390 The following are always recognized: 391 </para> 392 393 <table frame="all" colsep="1" rowsep="1"> 394 <title>Generic characters</title> 395 <tgroup cols="2"> 396 <colspec colnum="1" align="center"/> 397 <thead> 398 <row> 399 <entry>Escape</entry> 400 <entry>Meaning</entry> 401 </row> 402 </thead> 403 <tbody> 404 <row> 405 <entry>\d</entry> 406 <entry>any decimal digit</entry> 407 </row> 408 <row> 409 <entry>\D</entry> 410 <entry>any character that is not a decimal digit</entry> 411 </row> 412 <row> 413 <entry>\s</entry> 414 <entry>any whitespace character</entry> 415 </row> 416 <row> 417 <entry>\S</entry> 418 <entry>any character that is not a whitespace character</entry> 419 </row> 420 <row> 421 <entry>\w</entry> 422 <entry>any "word" character</entry> 423 </row> 424 <row> 425 <entry>\W</entry> 426 <entry>any "non-word" character</entry> 427 </row> 428 </tbody> 429 </tgroup> 430 </table> 431 432 <para> 433 Each pair of escape sequences partitions the complete set of characters 434 into two disjoint sets. Any given character matches one, and only one, 435 of each pair. 436 </para> 437 438 <para> 439 These character type sequences can appear both inside and outside character 440 classes. They each match one character of the appropriate type. 441 If the current matching point is at the end of the passed string, all 442 of them fail, since there is no character to match. 443 </para> 444 445 <para> 446 For compatibility with Perl, \s does not match the VT character (code 447 11). This makes it different from the the POSIX "space" class. The \s 448 characters are HT (9), LF (10), FF (12), CR (13), and space (32). 449 </para> 450 451 <para> 452 A "word" character is an underscore or any character less than 256 that 453 is a letter or digit.</para> 454 455 <para> 456 Characters with values greater than 128 never match \d, 457 \s, or \w, and always match \D, \S, and \W. 458 </para> 459 </refsect2> 460 461 <refsect2> 462 <title>Newline sequences</title> 463 <para>Outside a character class, the escape sequence \R matches any Unicode 464 newline sequence. 465 This particular group matches either the two-character sequence CR followed by 466 LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, 467 U+000B), FF (formfeed, U+000C), CR (carriage return, U+000D), NEL (next 468 line, U+0085), LS (line separator, U+2028), or PS (paragraph separator, U+2029). 469 The two-character sequence is treated as a single unit that 470 cannot be split. Inside a character class, \R matches the letter "R".</para> 471 </refsect2> 472 473 <refsect2> 474 <title>Unicode character properties</title> 475 <para> 476 To support generic character types there are three additional escape 477 sequences, they are: 478 </para> 479 480 <table frame="all" colsep="1" rowsep="1"> 481 <title>Generic character types</title> 482 <tgroup cols="2"> 483 <colspec colnum="1" align="center"/> 484 <thead> 485 <row> 486 <entry>Escape</entry> 487 <entry>Meaning</entry> 488 </row> 489 </thead> 490 <tbody> 491 <row> 492 <entry>\p{xx}</entry> 493 <entry>a character with the xx property</entry> 494 </row> 495 <row> 496 <entry>\P{xx}</entry> 497 <entry>a character without the xx property</entry> 498 </row> 499 <row> 500 <entry>\X</entry> 501 <entry>an extended Unicode sequence</entry> 502 </row> 503 </tbody> 504 </tgroup> 505 </table> 506 507 <para> 508 The property names represented by xx above are limited to the Unicode 509 script names, the general category properties, and "Any", which matches 510 any character (including newline). Other properties such as "InMusicalSymbols" 511 are not currently supported. Note that \P{Any} does not match any characters, 512 so always causes a match failure. 513 </para> 514 515 <para> 516 Sets of Unicode characters are defined as belonging to certain scripts. A 517 character from one of these sets can be matched using a script name. For 518 example, \p{Greek} or \P{Han}. 519 </para> 520 521 <para> 522 Those that are not part of an identified script are lumped together as 523 "Common". The current list of scripts is: 524 </para> 525 526 <itemizedlist> 527 <listitem><para>Arabic</para></listitem> 528 <listitem><para>Armenian</para></listitem> 529 <listitem><para>Balinese</para></listitem> 530 <listitem><para>Bengali</para></listitem> 531 <listitem><para>Bopomofo</para></listitem> 532 <listitem><para>Braille</para></listitem> 533 <listitem><para>Buginese</para></listitem> 534 <listitem><para>Buhid</para></listitem> 535 <listitem><para>Canadian_Aboriginal</para></listitem> 536 <listitem><para>Cherokee</para></listitem> 537 <listitem><para>Common</para></listitem> 538 <listitem><para>Coptic</para></listitem> 539 <listitem><para>Cuneiform</para></listitem> 540 <listitem><para>Cypriot</para></listitem> 541 <listitem><para>Cyrillic</para></listitem> 542 <listitem><para>Deseret</para></listitem> 543 <listitem><para>Devanagari</para></listitem> 544 <listitem><para>Ethiopic</para></listitem> 545 <listitem><para>Georgian</para></listitem> 546 <listitem><para>Glagolitic</para></listitem> 547 <listitem><para>Gothic</para></listitem> 548 <listitem><para>Greek</para></listitem> 549 <listitem><para>Gujarati</para></listitem> 550 <listitem><para>Gurmukhi</para></listitem> 551 <listitem><para>Han</para></listitem> 552 <listitem><para>Hangul</para></listitem> 553 <listitem><para>Hanunoo</para></listitem> 554 <listitem><para>Hebrew</para></listitem> 555 <listitem><para>Hiragana</para></listitem> 556 <listitem><para>Inherited</para></listitem> 557 <listitem><para>Kannada</para></listitem> 558 <listitem><para>Katakana</para></listitem> 559 <listitem><para>Kharoshthi</para></listitem> 560 <listitem><para>Khmer</para></listitem> 561 <listitem><para>Lao</para></listitem> 562 <listitem><para>Latin</para></listitem> 563 <listitem><para>Limbu</para></listitem> 564 <listitem><para>Linear_B</para></listitem> 565 <listitem><para>Malayalam</para></listitem> 566 <listitem><para>Mongolian</para></listitem> 567 <listitem><para>Myanmar</para></listitem> 568 <listitem><para>New_Tai_Lue</para></listitem> 569 <listitem><para>Nko</para></listitem> 570 <listitem><para>Ogham</para></listitem> 571 <listitem><para>Old_Italic</para></listitem> 572 <listitem><para>Old_Persian</para></listitem> 573 <listitem><para>Oriya</para></listitem> 574 <listitem><para>Osmanya</para></listitem> 575 <listitem><para>Phags_Pa</para></listitem> 576 <listitem><para>Phoenician</para></listitem> 577 <listitem><para>Runic</para></listitem> 578 <listitem><para>Shavian</para></listitem> 579 <listitem><para>Sinhala</para></listitem> 580 <listitem><para>Syloti_Nagri</para></listitem> 581 <listitem><para>Syriac</para></listitem> 582 <listitem><para>Tagalog</para></listitem> 583 <listitem><para>Tagbanwa</para></listitem> 584 <listitem><para>Tai_Le</para></listitem> 585 <listitem><para>Tamil</para></listitem> 586 <listitem><para>Telugu</para></listitem> 587 <listitem><para>Thaana</para></listitem> 588 <listitem><para>Thai</para></listitem> 589 <listitem><para>Tibetan</para></listitem> 590 <listitem><para>Tifinagh</para></listitem> 591 <listitem><para>Ugaritic</para></listitem> 592 <listitem><para>Yi</para></listitem> 593 </itemizedlist> 594 595 <para> 596 Each character has exactly one general category property, specified by a 597 two-letter abbreviation. For compatibility with Perl, negation can be specified 598 by including a circumflex between the opening brace and the property name. For 599 example, \p{^Lu} is the same as \P{Lu}. 600 </para> 601 602 <para> 603 If only one letter is specified with \p or \P, it includes all the general 604 category properties that start with that letter. In this case, in the absence 605 of negation, the curly brackets in the escape sequence are optional; these two 606 examples have the same effect: 607 </para> 608 609 <programlisting> 610 \p{L} 611 \pL 612 </programlisting> 613 614 <para> 615 The following general category property codes are supported: 616 </para> 617 618 <table frame="all" colsep="1" rowsep="1"> 619 <title>Property codes</title> 620 <tgroup cols="2"> 621 <colspec colnum="1" align="center"/> 622 <thead> 623 <row> 624 <entry>Code</entry> 625 <entry>Meaning</entry> 626 </row> 627 </thead> 628 <tbody> 629 <row> 630 <entry>C</entry> 631 <entry>Other</entry> 632 </row> 633 <row> 634 <entry>Cc</entry> 635 <entry>Control</entry> 636 </row> 637 <row> 638 <entry>Cf</entry> 639 <entry>Format</entry> 640 </row> 641 <row> 642 <entry>Cn</entry> 643 <entry>Unassigned</entry> 644 </row> 645 <row> 646 <entry>Co</entry> 647 <entry>Private use</entry> 648 </row> 649 <row> 650 <entry>Cs</entry> 651 <entry>Surrogate</entry> 652 </row> 653 <row> 654 <entry>L</entry> 655 <entry>Letter</entry> 656 </row> 657 <row> 658 <entry>Ll</entry> 659 <entry>Lower case letter</entry> 660 </row> 661 <row> 662 <entry>Lm</entry> 663 <entry>Modifier letter</entry> 664 </row> 665 <row> 666 <entry>Lo</entry> 667 <entry>Other letter</entry> 668 </row> 669 <row> 670 <entry>Lt</entry> 671 <entry>Title case letter</entry> 672 </row> 673 <row> 674 <entry>Lu</entry> 675 <entry>Upper case letter</entry> 676 </row> 677 <row> 678 <entry>M</entry> 679 <entry>Mark</entry> 680 </row> 681 <row> 682 <entry>Mc</entry> 683 <entry>Spacing mark</entry> 684 </row> 685 <row> 686 <entry>Me</entry> 687 <entry>Enclosing mark</entry> 688 </row> 689 <row> 690 <entry>Mn</entry> 691 <entry>Non-spacing mark</entry> 692 </row> 693 <row> 694 <entry>N</entry> 695 <entry>Number</entry> 696 </row> 697 <row> 698 <entry>Nd</entry> 699 <entry>Decimal number</entry> 700 </row> 701 <row> 702 <entry>Nl</entry> 703 <entry>Letter number</entry> 704 </row> 705 <row> 706 <entry>No</entry> 707 <entry>Other number</entry> 708 </row> 709 <row> 710 <entry>P</entry> 711 <entry>Punctuation</entry> 712 </row> 713 <row> 714 <entry>Pc</entry> 715 <entry>Connector punctuation</entry> 716 </row> 717 <row> 718 <entry>Pd</entry> 719 <entry>Dash punctuation</entry> 720 </row> 721 <row> 722 <entry>Pe</entry> 723 <entry>Close punctuation</entry> 724 </row> 725 <row> 726 <entry>Pf</entry> 727 <entry>Final punctuation</entry> 728 </row> 729 <row> 730 <entry>Pi</entry> 731 <entry>Initial punctuation</entry> 732 </row> 733 <row> 734 <entry>Po</entry> 735 <entry>Other punctuation</entry> 736 </row> 737 <row> 738 <entry>Ps</entry> 739 <entry>Open punctuation</entry> 740 </row> 741 <row> 742 <entry>S</entry> 743 <entry>Symbol</entry> 744 </row> 745 <row> 746 <entry>Sc</entry> 747 <entry>Currency symbol</entry> 748 </row> 749 <row> 750 <entry>Sk</entry> 751 <entry>Modifier symbol</entry> 752 </row> 753 <row> 754 <entry>Sm</entry> 755 <entry>Mathematical symbol</entry> 756 </row> 757 <row> 758 <entry>So</entry> 759 <entry>Other symbol</entry> 760 </row> 761 <row> 762 <entry>Z</entry> 763 <entry>Separator</entry> 764 </row> 765 <row> 766 <entry>Zl</entry> 767 <entry>Line separator</entry> 768 </row> 769 <row> 770 <entry>Zp</entry> 771 <entry>Paragraph separator</entry> 772 </row> 773 <row> 774 <entry>Zs</entry> 775 <entry>Space separator</entry> 776 </row> 777 </tbody> 778 </tgroup> 779 </table> 780 781 <para> 782 The special property L& is also supported: it matches a character that has 783 the Lu, Ll, or Lt property, in other words, a letter that is not classified as 784 a modifier or "other". 785 </para> 786 787 <para> 788 The long synonyms for these properties that Perl supports (such as \ep{Letter}) 789 are not supported by GRegex, nor is it permitted to prefix any of these 790 properties with "Is". 791 </para> 792 793 <para> 794 No character that is in the Unicode table has the Cn (unassigned) property. 795 Instead, this property is assumed for any code point that is not in the 796 Unicode table. 797 </para> 798 799 <para> 800 Specifying caseless matching does not affect these escape sequences. 801 For example, \p{Lu} always matches only upper case letters. 802 </para> 803 804 <para> 805 The \X escape matches any number of Unicode characters that form an 806 extended Unicode sequence. \X is equivalent to 807 </para> 808 809 <programlisting> 810 (?>\PM\pM*) 811 </programlisting> 812 813 <para> 814 That is, it matches a character without the "mark" property, followed 815 by zero or more characters with the "mark" property, and treats the 816 sequence as an atomic group (see below). Characters with the "mark" 817 property are typically accents that affect the preceding character. 818 </para> 819 820 <para> 821 Matching characters by Unicode property is not fast, because GRegex has 822 to search a structure that contains data for over fifteen thousand 823 characters. That is why the traditional escape sequences such as \d and 824 \w do not use Unicode properties. 825 </para> 826 </refsect2> 827 828 <refsect2> 829 <title>Simple assertions</title> 830 <para> 831 The final use of backslash is for certain simple assertions. An 832 assertion specifies a condition that has to be met at a particular point in 833 a match, without consuming any characters from the string. The 834 use of subpatterns for more complicated assertions is described below. 835 The backslashed assertions are: 836 </para> 837 838 <table frame="all" colsep="1" rowsep="1"> 839 <title>Simple assertions</title> 840 <tgroup cols="2"> 841 <colspec colnum="1" align="center"/> 842 <thead> 843 <row> 844 <entry>Escape</entry> 845 <entry>Meaning</entry> 846 </row> 847 </thead> 848 <tbody> 849 <row> 850 <entry>\b</entry> 851 <entry>matches at a word boundary</entry> 852 </row> 853 <row> 854 <entry>\B</entry> 855 <entry>matches when not at a word boundary</entry> 856 </row> 857 <row> 858 <entry>\A</entry> 859 <entry>matches at the start of the string</entry> 860 </row> 861 <row> 862 <entry>\Z</entry> 863 <entry>matches at the end of the string or before a newline at the end of the string</entry> 864 </row> 865 <row> 866 <entry>\z</entry> 867 <entry>matches only at the end of the string</entry> 868 </row> 869 <row> 870 <entry>\G</entry> 871 <entry>matches at first matching position in the string</entry> 872 </row> 873 </tbody> 874 </tgroup> 875 </table> 876 877 <para> 878 These assertions may not appear in character classes (but note that \b 879 has a different meaning, namely the backspace character, inside a 880 character class). 881 </para> 882 883 <para> 884 A word boundary is a position in the string where the current 885 character and the previous character do not both match \w or \W (i.e. 886 one matches \w and the other matches \W), or the start or end of the 887 string if the first or last character matches \w, respectively. 888 </para> 889 890 <para> 891 The \A, \Z, and \z assertions differ from the traditional circumflex 892 and dollar (described in the next section) in that they only ever match 893 at the very start and end of the string, whatever options are 894 set. Thus, they are independent of multiline mode. These three assertions 895 are not affected by the <varname>G_REGEX_MATCH_NOTBOL</varname> or <varname>G_REGEX_MATCH_NOTEOL</varname> options, 896 which affect only the behaviour of the circumflex and dollar metacharacters. 897 However, if the start_position argument of a matching function is non-zero, 898 indicating that matching is to start at a point other than the beginning of 899 the string, \A can never match. The difference between \Z and \z is 900 that \Z matches before a newline at the end of the string as well at the 901 very end, whereas \z matches only at the end. 902 </para> 903 904 <para> 905 The \G assertion is true only when the current matching position is at 906 the start point of the match, as specified by the start_position argument 907 to the matching functions. It differs from \A when the value of startoffset is 908 non-zero. 909 </para> 910 911 <para> 912 Note, however, that the interpretation of \G, as the start of the 913 current match, is subtly different from Perls, which defines it as the 914 end of the previous match. In Perl, these can be different when the 915 previously matched string was empty. 916 </para> 917 918 <para> 919 If all the alternatives of a pattern begin with \G, the expression is 920 anchored to the starting match position, and the "anchored" flag is set 921 in the compiled regular expression. 922 </para> 923 </refsect2> 924 </refsect1> 925 926 <refsect1> 927 <title>Circumflex and dollar</title> 928 <para> 929 Outside a character class, in the default matching mode, the circumflex 930 character is an assertion that is true only if the current matching 931 point is at the start of the string. If the start_position argument to 932 the matching functions is non-zero, circumflex can never match if the 933 <varname>G_REGEX_MULTILINE</varname> option is unset. Inside a character class, circumflex 934 has an entirely different meaning (see below). 935 </para> 936 937 <para> 938 Circumflex need not be the first character of the pattern if a number 939 of alternatives are involved, but it should be the first thing in each 940 alternative in which it appears if the pattern is ever to match that 941 branch. If all possible alternatives start with a circumflex, that is, 942 if the pattern is constrained to match only at the start of the string, 943 it is said to be an "anchored" pattern. (There are also other 944 constructs that can cause a pattern to be anchored.) 945 </para> 946 947 <para> 948 A dollar character is an assertion that is true only if the current 949 matching point is at the end of the string, or immediately 950 before a newline at the end of the string (by default). Dollar need not 951 be the last character of the pattern if a number of alternatives are 952 involved, but it should be the last item in any branch in which it 953 appears. Dollar has no special meaning in a character class. 954 </para> 955 956 <para> 957 The meaning of dollar can be changed so that it matches only at the 958 very end of the string, by setting the <varname>G_REGEX_DOLLAR_ENDONLY</varname> option at 959 compile time. This does not affect the \Z assertion. 960 </para> 961 962 <para> 963 The meanings of the circumflex and dollar characters are changed if the 964 <varname>G_REGEX_MULTILINE</varname> option is set. When this is the case, 965 a circumflex matches immediately after internal newlines as well as at the 966 start of the string. It does not match after a newline that ends the string. 967 A dollar matches before any newlines in the string, as well as at the very 968 end, when <varname>G_REGEX_MULTILINE</varname> is set. When newline is 969 specified as the two-character sequence CRLF, isolated CR and LF characters 970 do not indicate newlines. 971 </para> 972 973 <para> 974 For example, the pattern /^abc$/ matches the string "def\nabc" (where 975 \n represents a newline) in multiline mode, but not otherwise. Consequently, 976 patterns that are anchored in single line mode because all branches start with 977 ^ are not anchored in multiline mode, and a match for circumflex is possible 978 when the <varname>start_position</varname> argument of a matching function 979 is non-zero. The <varname>G_REGEX_DOLLAR_ENDONLY</varname> option is ignored 980 if <varname>G_REGEX_MULTILINE</varname> is set. 981 </para> 982 983 <para> 984 Note that the sequences \A, \Z, and \z can be used to match the start and 985 end of the string in both modes, and if all branches of a pattern start with 986 \A it is always anchored, whether or not <varname>G_REGEX_MULTILINE</varname> 987 is set. 988 </para> 989 </refsect1> 990 991 <refsect1> 992 <title>Full stop (period, dot)</title> 993 <para> 994 Outside a character class, a dot in the pattern matches any one character 995 in the string, including a non-printing character, but not (by 996 default) newline. In UTF-8 a character might be more than one byte long. 997 </para> 998 999 <para> 1000 When a line ending is defined as a single character, dot never matches that 1001 character; when the two-character sequence CRLF is used, dot does not match CR 1002 if it is immediately followed by LF, but otherwise it matches all characters 1003 (including isolated CRs and LFs). When any Unicode line endings are being 1004 recognized, dot does not match CR or LF or any of the other line ending 1005 characters. 1006 </para> 1007 1008 <para> 1009 If the <varname>G_REGEX_DOTALL</varname> flag is set, dots match newlines 1010 as well. The handling of dot is entirely independent of the handling of circumflex 1011 and dollar, the only relationship being that they both involve newline 1012 characters. Dot has no special meaning in a character class. 1013 </para> 1014 1015 <para> 1016 The behaviour of dot with regard to newlines can be changed. If the 1017 <varname>G_REGEX_DOTALL</varname> option is set, a dot matches any one 1018 character, without exception. If newline is defined as the two-character 1019 sequence CRLF, it takes two dots to match it. 1020 </para> 1021 1022 <para> 1023 The handling of dot is entirely independent of the handling of circumflex and 1024 dollar, the only relationship being that they both involve newlines. Dot has no 1025 special meaning in a character class. 1026 </para> 1027 </refsect1> 1028 1029 <refsect1> 1030 <title>Matching a single byte</title> 1031 <para> 1032 Outside a character class, the escape sequence \C matches any one byte, 1033 both in and out of UTF-8 mode. Unlike a dot, it always matches any line 1034 ending characters. 1035 The feature is provided in Perl in order to match individual bytes in 1036 UTF-8 mode. Because it breaks up UTF-8 characters into individual 1037 bytes, what remains in the string may be a malformed UTF-8 string. For 1038 this reason, the \C escape sequence is best avoided. 1039 </para> 1040 1041 <para> 1042 GRegex does not allow \C to appear in lookbehind assertions (described 1043 below), because in UTF-8 mode this would make it impossible to calculate 1044 the length of the lookbehind. 1045 </para> 1046 </refsect1> 1047 1048 <refsect1> 1049 <title>Square brackets and character classes</title> 1050 <para> 1051 An opening square bracket introduces a character class, terminated by a 1052 closing square bracket. A closing square bracket on its own is not special. If a closing square bracket is required as a member of the class, 1053 it should be the first data character in the class (after an initial 1054 circumflex, if present) or escaped with a backslash. 1055 </para> 1056 1057 <para> 1058 A character class matches a single character in the string. A matched character 1059 must be in the set of characters defined by the class, unless the first 1060 character in the class definition is a circumflex, in which case the 1061 string character must not be in the set defined by the class. If a 1062 circumflex is actually required as a member of the class, ensure it is 1063 not the first character, or escape it with a backslash. 1064 </para> 1065 1066 <para> 1067 For example, the character class [aeiou] matches any lower case vowel, 1068 while [^aeiou] matches any character that is not a lower case vowel. 1069 Note that a circumflex is just a convenient notation for specifying the 1070 characters that are in the class by enumerating those that are not. A 1071 class that starts with a circumflex is not an assertion: it still consumes 1072 a character from the string, and therefore it fails if the current pointer 1073 is at the end of the string. 1074 </para> 1075 1076 <para> 1077 In UTF-8 mode, characters with values greater than 255 can be included 1078 in a class as a literal string of bytes, or by using the \x{ escaping 1079 mechanism. 1080 </para> 1081 1082 <para> 1083 When caseless matching is set, any letters in a class represent both 1084 their upper case and lower case versions, so for example, a caseless 1085 [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not 1086 match "A", whereas a caseful version would. 1087 </para> 1088 1089 <para> 1090 Characters that might indicate line breaks are never treated 1091 in any special way when matching character classes, whatever line-ending 1092 sequence is in use, and whatever setting of the <varname>G_REGEX_DOTALL</varname> 1093 and <varname>G_REGEX_MULTILINE</varname> options is used. A class such as [^a] 1094 always matches one of these characters. 1095 </para> 1096 1097 <para> 1098 The minus (hyphen) character can be used to specify a range of characters in 1099 a character class. For example, [d-m] matches any letter 1100 between d and m, inclusive. If a minus character is required in a 1101 class, it must be escaped with a backslash or appear in a position 1102 where it cannot be interpreted as indicating a range, typically as the 1103 first or last character in the class. 1104 </para> 1105 1106 <para> 1107 It is not possible to have the literal character "]" as the end character 1108 of a range. A pattern such as [W-]46] is interpreted as a class of 1109 two characters ("W" and "-") followed by a literal string "46]", so it 1110 would match "W46]" or "-46]". However, if the "]" is escaped with a 1111 backslash it is interpreted as the end of range, so [W-\]46] is interpreted 1112 as a class containing a range followed by two other characters. 1113 The octal or hexadecimal representation of "]" can also be used to end 1114 a range. 1115 </para> 1116 1117 <para> 1118 Ranges operate in the collating sequence of character values. They can 1119 also be used for characters specified numerically, for example 1120 [\000-\037]. In UTF-8 mode, ranges can include characters whose values 1121 are greater than 255, for example [\x{100}-\x{2ff}]. 1122 </para> 1123 1124 <para> 1125 The character types \d, \D, \p, \P, \s, \S, \w, and \W may also appear 1126 in a character class, and add the characters that they match to the 1127 class. For example, [\dABCDEF] matches any hexadecimal digit. A 1128 circumflex can conveniently be used with the upper case character types to 1129 specify a more restricted set of characters than the matching lower 1130 case type. For example, the class [^\W_] matches any letter or digit, 1131 but not underscore. 1132 </para> 1133 1134 <para> 1135 The only metacharacters that are recognized in character classes are 1136 backslash, hyphen (only where it can be interpreted as specifying a 1137 range), circumflex (only at the start), opening square bracket (only 1138 when it can be interpreted as introducing a POSIX class name - see the 1139 next section), and the terminating closing square bracket. However, 1140 escaping other non-alphanumeric characters does no harm. 1141 </para> 1142 </refsect1> 1143 1144 <refsect1> 1145 <title>Posix character classes</title> 1146 <para> 1147 GRegex supports the POSIX notation for character classes. This uses names 1148 enclosed by [: and :] within the enclosing square brackets. For example, 1149 </para> 1150 1151 <programlisting> 1152 [01[:alpha:]%] 1153 </programlisting> 1154 1155 <para> 1156 matches "0", "1", any alphabetic character, or "%". The supported class 1157 names are 1158 </para> 1159 1160 <table frame="all" colsep="1" rowsep="1"> 1161 <title>Posix classes</title> 1162 <tgroup cols="2"> 1163 <colspec colnum="1" align="center"/> 1164 <thead> 1165 <row> 1166 <entry>Name</entry> 1167 <entry>Meaning</entry> 1168 </row> 1169 </thead> 1170 <tbody> 1171 <row> 1172 <entry>alnum</entry> 1173 <entry>letters and digits</entry> 1174 </row> 1175 <row> 1176 <entry>alpha</entry> 1177 <entry>letters</entry> 1178 </row> 1179 <row> 1180 <entry>ascii</entry> 1181 <entry>character codes 0 - 127</entry> 1182 </row> 1183 <row> 1184 <entry>blank</entry> 1185 <entry>space or tab only</entry> 1186 </row> 1187 <row> 1188 <entry>cntrl</entry> 1189 <entry>control characters</entry> 1190 </row> 1191 <row> 1192 <entry>digit</entry> 1193 <entry>decimal digits (same as \d)</entry> 1194 </row> 1195 <row> 1196 <entry>graph</entry> 1197 <entry>printing characters, excluding space</entry> 1198 </row> 1199 <row> 1200 <entry>lower</entry> 1201 <entry>lower case letters</entry> 1202 </row> 1203 <row> 1204 <entry>print</entry> 1205 <entry>printing characters, including space</entry> 1206 </row> 1207 <row> 1208 <entry>punct</entry> 1209 <entry>printing characters, excluding letters and digits</entry> 1210 </row> 1211 <row> 1212 <entry>space</entry> 1213 <entry>white space (not quite the same as \s)</entry> 1214 </row> 1215 <row> 1216 <entry>upper</entry> 1217 <entry>upper case letters</entry> 1218 </row> 1219 <row> 1220 <entry>word</entry> 1221 <entry>"word" characters (same as \w)</entry> 1222 </row> 1223 <row> 1224 <entry>xdigit</entry> 1225 <entry>hexadecimal digits</entry> 1226 </row> 1227 </tbody> 1228 </tgroup> 1229 </table> 1230 1231 <para> 1232 The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), 1233 and space (32). Notice that this list includes the VT character (code 1234 11). This makes "space" different to \s, which does not include VT (for 1235 Perl compatibility). 1236 </para> 1237 1238 <para> 1239 The name "word" is a Perl extension, and "blank" is a GNU extension. 1240 Another Perl extension is negation, which is indicated by a ^ character 1241 after the colon. For example, 1242 </para> 1243 1244 <programlisting> 1245 [12[:^digit:]] 1246 </programlisting> 1247 1248 <para> 1249 matches "1", "2", or any non-digit. GRegex also recognize the 1250 POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but 1251 these are not supported, and an error is given if they are encountered. 1252 </para> 1253 1254 <para> 1255 In UTF-8 mode, characters with values greater than 128 do not match any 1256 of the POSIX character classes. 1257 </para> 1258 </refsect1> 1259 1260 <refsect1> 1261 <title>Vertical bar</title> 1262 <para> 1263 Vertical bar characters are used to separate alternative patterns. For 1264 example, the pattern 1265 </para> 1266 1267 <programlisting> 1268 gilbert|sullivan 1269 </programlisting> 1270 1271 <para> 1272 matches either "gilbert" or "sullivan". Any number of alternatives may 1273 appear, and an empty alternative is permitted (matching the empty 1274 string). The matching process tries each alternative in turn, from 1275 left to right, and the first one that succeeds is used. If the alternatives are within a subpattern (defined below), "succeeds" means matching the rest of the main pattern as well as the alternative in the subpattern. 1276 </para> 1277 </refsect1> 1278 1279 <refsect1> 1280 <title>Internal option setting</title> 1281 <para> 1282 The settings of the <varname>G_REGEX_CASELESS</varname>, <varname>G_REGEX_MULTILINE</varname>, <varname>G_REGEX_MULTILINE</varname>, 1283 and <varname>G_REGEX_EXTENDED</varname> options can be changed from within the pattern by a 1284 sequence of Perl-style option letters enclosed between "(?" and ")". The 1285 option letters are 1286 </para> 1287 1288 <table frame="all" colsep="1" rowsep="1"> 1289 <title>Option settings</title> 1290 <tgroup cols="2"> 1291 <colspec colnum="1" align="center"/> 1292 <thead> 1293 <row> 1294 <entry>Option</entry> 1295 <entry>Flag</entry> 1296 </row> 1297 </thead> 1298 <tbody> 1299 <row> 1300 <entry>i</entry> 1301 <entry><varname>G_REGEX_CASELESS</varname></entry> 1302 </row> 1303 <row> 1304 <entry>m</entry> 1305 <entry><varname>G_REGEX_MULTILINE</varname></entry> 1306 </row> 1307 <row> 1308 <entry>s</entry> 1309 <entry><varname>G_REGEX_DOTALL</varname></entry> 1310 </row> 1311 <row> 1312 <entry>x</entry> 1313 <entry><varname>G_REGEX_EXTENDED</varname></entry> 1314 </row> 1315 </tbody> 1316 </tgroup> 1317 </table> 1318 1319 <para> 1320 For example, (?im) sets caseless, multiline matching. It is also 1321 possible to unset these options by preceding the letter with a hyphen, and a 1322 combined setting and unsetting such as (?im-sx), which sets <varname>G_REGEX_CASELESS</varname> 1323 and <varname>G_REGEX_MULTILINE</varname> while unsetting <varname>G_REGEX_DOTALL</varname> and <varname>G_REGEX_EXTENDED</varname>, 1324 is also permitted. If a letter appears both before and after the 1325 hyphen, the option is unset. 1326 </para> 1327 1328 <para> 1329 When an option change occurs at top level (that is, not inside subpattern 1330 parentheses), the change applies to the remainder of the pattern 1331 that follows. 1332 </para> 1333 1334 <para> 1335 An option change within a subpattern (see below for a description of subpatterns) 1336 affects only that part of the current pattern that follows it, so 1337 </para> 1338 1339 <programlisting> 1340 (a(?i)b)c 1341 </programlisting> 1342 1343 <para> 1344 matches abc and aBc and no other strings (assuming <varname>G_REGEX_CASELESS</varname> is not 1345 used). By this means, options can be made to have different settings 1346 in different parts of the pattern. Any changes made in one alternative 1347 do carry on into subsequent branches within the same subpattern. For 1348 example, 1349 </para> 1350 1351 <programlisting> 1352 (a(?i)b|c) 1353 </programlisting> 1354 1355 <para> 1356 matches "ab", "aB", "c", and "C", even though when matching "C" the 1357 first branch is abandoned before the option setting. This is because 1358 the effects of option settings happen at compile time. There would be 1359 some very weird behaviour otherwise. 1360 </para> 1361 1362 <para> 1363 The options <varname>G_REGEX_UNGREEDY</varname> and 1364 <varname>G_REGEX_EXTRA</varname> and <varname>G_REGEX_DUPNAMES</varname> 1365 can be changed in the same way as the Perl-compatible options by using 1366 the characters U, X and J respectively. 1367 </para> 1368 </refsect1> 1369 1370 <refsect1> 1371 <title>Subpatterns</title> 1372 <para> 1373 Subpatterns are delimited by parentheses (round brackets), which can be 1374 nested. Turning part of a pattern into a subpattern does two things: 1375 </para> 1376 1377 <itemizedlist> 1378 <listitem><para> 1379 It localizes a set of alternatives. For example, the pattern 1380 cat(aract|erpillar|) matches one of the words "cat", "cataract", or 1381 "caterpillar". Without the parentheses, it would match "cataract", 1382 "erpillar" or an empty string. 1383 </para></listitem> 1384 <listitem><para> 1385 It sets up the subpattern as a capturing subpattern. This means 1386 that, when the whole pattern matches, that portion of the 1387 string that matched the subpattern can be obtained using <function>g_regex_fetch()</function>. 1388 Opening parentheses are counted from left to right (starting from 1, as 1389 subpattern 0 is the whole matched string) to obtain numbers for the 1390 capturing subpatterns. 1391 </para></listitem> 1392 </itemizedlist> 1393 1394 <para> 1395 For example, if the string "the red king" is matched against the pattern 1396 </para> 1397 1398 <programlisting> 1399 the ((red|white) (king|queen)) 1400 </programlisting> 1401 1402 <para> 1403 the captured substrings are "red king", "red", and "king", and are numbered 1, 2, and 3, respectively. 1404 </para> 1405 1406 <para> 1407 The fact that plain parentheses fulfil two functions is not always 1408 helpful. There are often times when a grouping subpattern is required 1409 without a capturing requirement. If an opening parenthesis is followed 1410 by a question mark and a colon, the subpattern does not do any capturing, 1411 and is not counted when computing the number of any subsequent 1412 capturing subpatterns. For example, if the string "the white queen" is 1413 matched against the pattern 1414 </para> 1415 1416 <programlisting> 1417 the ((?:red|white) (king|queen)) 1418 </programlisting> 1419 1420 <para> 1421 the captured substrings are "white queen" and "queen", and are numbered 1422 1 and 2. The maximum number of capturing subpatterns is 65535. 1423 </para> 1424 1425 <para> 1426 As a convenient shorthand, if any option settings are required at the 1427 start of a non-capturing subpattern, the option letters may appear 1428 between the "?" and the ":". Thus the two patterns 1429 </para> 1430 1431 <programlisting> 1432 (?i:saturday|sunday) 1433 (?:(?i)saturday|sunday) 1434 </programlisting> 1435 1436 <para> 1437 match exactly the same set of strings. Because alternative branches are 1438 tried from left to right, and options are not reset until the end of 1439 the subpattern is reached, an option setting in one branch does affect 1440 subsequent branches, so the above patterns match "SUNDAY" as well as 1441 "Saturday". 1442 </para> 1443 </refsect1> 1444 1445 <refsect1> 1446 <title>Named subpatterns</title> 1447 <para> 1448 Identifying capturing parentheses by number is simple, but it can be 1449 very hard to keep track of the numbers in complicated regular expressions. 1450 Furthermore, if an expression is modified, the numbers may 1451 change. To help with this difficulty, GRegex supports the naming of 1452 subpatterns. A subpattern can be named in one of three ways: (?<name>...) or 1453 (?'name'...) as in Perl, or (?P<name>...) as in Python. 1454 References to capturing parentheses from other 1455 parts of the pattern, such as backreferences, recursion, and conditions, 1456 can be made by name as well as by number. 1457 </para> 1458 1459 <para> 1460 Names consist of up to 32 alphanumeric characters and underscores. Named 1461 capturing parentheses are still allocated numbers as well as names, exactly as 1462 if the names were not present. 1463 By default, a name must be unique within a pattern, but it is possible to relax 1464 this constraint by setting the <varname>G_REGEX_DUPNAMES</varname> option at 1465 compile time. This can be useful for patterns where only one instance of the 1466 named parentheses can match. Suppose you want to match the name of a weekday, 1467 either as a 3-letter abbreviation or as the full name, and in both cases you 1468 want to extract the abbreviation. This pattern (ignoring the line breaks) does 1469 the job: 1470 </para> 1471 1472 <programlisting> 1473 (?<DN>Mon|Fri|Sun)(?:day)?| 1474 (?<DN>Tue)(?:sday)?| 1475 (?<DN>Wed)(?:nesday)?| 1476 (?<DN>Thu)(?:rsday)?| 1477 (?<DN>Sat)(?:urday)? 1478 </programlisting> 1479 1480 <para> 1481 There are five capturing substrings, but only one is ever set after a match. 1482 The function for extracting the data by name returns the substring 1483 for the first (and in this example, the only) subpattern of that name that 1484 matched. This saves searching to find which numbered subpattern it was. If you 1485 make a reference to a non-unique named subpattern from elsewhere in the 1486 pattern, the one that corresponds to the lowest number is used. 1487 </para> 1488 </refsect1> 1489 1490 <refsect1> 1491 <title>Repetition</title> 1492 <para> 1493 Repetition is specified by quantifiers, which can follow any of the 1494 following items: 1495 </para> 1496 1497 <itemizedlist> 1498 <listitem><para>a literal data character</para></listitem> 1499 <listitem><para>the dot metacharacter</para></listitem> 1500 <listitem><para>the \C escape sequence</para></listitem> 1501 <listitem><para>the \X escape sequence (in UTF-8 mode)</para></listitem> 1502 <listitem><para>the \R escape sequence</para></listitem> 1503 <listitem><para>an escape such as \d that matches a single character</para></listitem> 1504 <listitem><para>a character class</para></listitem> 1505 <listitem><para>a back reference (see next section)</para></listitem> 1506 <listitem><para>a parenthesized subpattern (unless it is an assertion)</para></listitem> 1507 </itemizedlist> 1508 1509 <para> 1510 The general repetition quantifier specifies a minimum and maximum number 1511 of permitted matches, by giving the two numbers in curly brackets 1512 (braces), separated by a comma. The numbers must be less than 65536, 1513 and the first must be less than or equal to the second. For example: 1514 </para> 1515 1516 <programlisting> 1517 z{2,4} 1518 </programlisting> 1519 1520 <para> 1521 matches "zz", "zzz", or "zzzz". A closing brace on its own is not a 1522 special character. If the second number is omitted, but the comma is 1523 present, there is no upper limit; if the second number and the comma 1524 are both omitted, the quantifier specifies an exact number of required 1525 matches. Thus 1526 </para> 1527 1528 <programlisting> 1529 [aeiou]{3,} 1530 </programlisting> 1531 1532 <para> 1533 matches at least 3 successive vowels, but may match many more, while 1534 </para> 1535 1536 <programlisting> 1537 \d{8} 1538 </programlisting> 1539 1540 <para> 1541 matches exactly 8 digits. An opening curly bracket that appears in a 1542 position where a quantifier is not allowed, or one that does not match 1543 the syntax of a quantifier, is taken as a literal character. For example, 1544 {,6} is not a quantifier, but a literal string of four characters. 1545 </para> 1546 1547 <para> 1548 In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to 1549 individual bytes. Thus, for example, \x{100}{2} matches two UTF-8 1550 characters, each of which is represented by a two-byte sequence. Similarly, 1551 \X{3} matches three Unicode extended sequences, each of which may be 1552 several bytes long (and they may be of different lengths). 1553 </para> 1554 1555 <para> 1556 The quantifier {0} is permitted, causing the expression to behave as if 1557 the previous item and the quantifier were not present. 1558 </para> 1559 1560 <para> 1561 For convenience, the three most common quantifiers have single-character 1562 abbreviations: 1563 </para> 1564 1565 <table frame="all" colsep="1" rowsep="1"> 1566 <title>Abbreviations for quantifiers</title> 1567 <tgroup cols="2"> 1568 <colspec colnum="1" align="center"/> 1569 <thead> 1570 <row> 1571 <entry>Abbreviation</entry> 1572 <entry>Meaning</entry> 1573 </row> 1574 </thead> 1575 <tbody> 1576 <row> 1577 <entry>*</entry> 1578 <entry>is equivalent to {0,}</entry> 1579 </row> 1580 <row> 1581 <entry>+</entry> 1582 <entry>is equivalent to {1,}</entry> 1583 </row> 1584 <row> 1585 <entry>?</entry> 1586 <entry>is equivalent to {0,1}</entry> 1587 </row> 1588 </tbody> 1589 </tgroup> 1590 </table> 1591 1592 <para> 1593 It is possible to construct infinite loops by following a subpattern 1594 that can match no characters with a quantifier that has no upper limit, 1595 for example: 1596 </para> 1597 1598 <programlisting> 1599 (a?)* 1600 </programlisting> 1601 1602 <para> 1603 Because there are cases where this can be useful, such patterns are 1604 accepted, but if any repetition of the subpattern does in fact match 1605 no characters, the loop is forcibly broken. 1606 </para> 1607 1608 <para> 1609 By default, the quantifiers are "greedy", that is, they match as much 1610 as possible (up to the maximum number of permitted times), without 1611 causing the rest of the pattern to fail. The classic example of where 1612 this gives problems is in trying to match comments in C programs. These 1613 appear between /* and */ and within the comment, individual * and / 1614 characters may appear. An attempt to match C comments by applying the 1615 pattern 1616 </para> 1617 1618 <programlisting> 1619 /\*.*\*/ 1620 </programlisting> 1621 1622 <para> 1623 to the string 1624 </para> 1625 1626 <programlisting> 1627 /* first comment */ not comment /* second comment */ 1628 </programlisting> 1629 1630 <para> 1631 fails, because it matches the entire string owing to the greediness of 1632 the .* item. 1633 </para> 1634 1635 <para> 1636 However, if a quantifier is followed by a question mark, it ceases to 1637 be greedy, and instead matches the minimum number of times possible, so 1638 the pattern 1639 </para> 1640 1641 <programlisting> 1642 /\*.*?\*/ 1643 </programlisting> 1644 1645 <para> 1646 does the right thing with the C comments. The meaning of the various 1647 quantifiers is not otherwise changed, just the preferred number of 1648 matches. Do not confuse this use of question mark with its use as a 1649 quantifier in its own right. Because it has two uses, it can sometimes 1650 appear doubled, as in 1651 </para> 1652 1653 <programlisting> 1654 \d??\d 1655 </programlisting> 1656 1657 <para> 1658 which matches one digit by preference, but can match two if that is the 1659 only way the rest of the pattern matches. 1660 </para> 1661 1662 <para> 1663 If the <varname>G_REGEX_UNGREEDY</varname> flag is set, the quantifiers are not greedy 1664 by default, but individual ones can be made greedy by following them with 1665 a question mark. In other words, it inverts the default behaviour. 1666 </para> 1667 1668 <para> 1669 When a parenthesized subpattern is quantified with a minimum repeat 1670 count that is greater than 1 or with a limited maximum, more memory is 1671 required for the compiled pattern, in proportion to the size of the 1672 minimum or maximum. 1673 </para> 1674 1675 <para> 1676 If a pattern starts with .* or .{0,} and the <varname>G_REGEX_DOTALL</varname> flag 1677 is set, thus allowing the dot to match newlines, the 1678 pattern is implicitly anchored, because whatever follows will be tried 1679 against every character position in the string, so there is no 1680 point in retrying the overall match at any position after the first. 1681 GRegex normally treats such a pattern as though it were preceded by \A. 1682 </para> 1683 1684 <para> 1685 In cases where it is known that the string contains no newlines, it 1686 is worth setting <varname>G_REGEX_DOTALL</varname> in order to obtain this optimization, 1687 or alternatively using ^ to indicate anchoring explicitly. 1688 </para> 1689 1690 <para> 1691 However, there is one situation where the optimization cannot be used. 1692 When .* is inside capturing parentheses that are the subject of a 1693 backreference elsewhere in the pattern, a match at the start may fail 1694 where a later one succeeds. Consider, for example: 1695 </para> 1696 1697 <programlisting> 1698 (.*)abc\1 1699 </programlisting> 1700 1701 <para> 1702 If the string is "xyz123abc123" the match point is the fourth character. 1703 For this reason, such a pattern is not implicitly anchored. 1704 </para> 1705 1706 <para> 1707 When a capturing subpattern is repeated, the value captured is the 1708 substring that matched the final iteration. For example, after 1709 </para> 1710 1711 <programlisting> 1712 (tweedle[dume]{3}\s*)+ 1713 </programlisting> 1714 1715 <para> 1716 has matched "tweedledum tweedledee" the value of the captured substring 1717 is "tweedledee". However, if there are nested capturing subpatterns, 1718 the corresponding captured values may have been set in previous iterations. 1719 For example, after 1720 </para> 1721 1722 <programlisting> 1723 /(a|(b))+/ 1724 </programlisting> 1725 1726 <para> 1727 matches "aba" the value of the second captured substring is "b". 1728 </para> 1729 </refsect1> 1730 1731 <refsect1> 1732 <title>Atomic grouping and possessive quantifiers</title> 1733 <para> 1734 With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") 1735 repetition, failure of what follows normally causes the repeated 1736 item to be re-evaluated to see if a different number 1737 of repeats allows the rest of the pattern to match. Sometimes it 1738 is useful to prevent this, either to change the nature of the 1739 match, or to cause it fail earlier than it otherwise might, when the 1740 author of the pattern knows there is no point in carrying on. 1741 </para> 1742 1743 <para> 1744 Consider, for example, the pattern \d+foo when applied to the string 1745 </para> 1746 1747 <programlisting> 1748 123456bar 1749 </programlisting> 1750 1751 <para> 1752 After matching all 6 digits and then failing to match "foo", the normal 1753 action of the matcher is to try again with only 5 digits matching the 1754 \d+ item, and then with 4, and so on, before ultimately failing. 1755 "Atomic grouping" (a term taken from Jeffrey Friedls book) provides 1756 the means for specifying that once a subpattern has matched, it is not 1757 to be re-evaluated in this way. 1758 </para> 1759 1760 <para> 1761 If we use atomic grouping for the previous example, the matcher 1762 give up immediately on failing to match "foo" the first time. The notation 1763 is a kind of special parenthesis, starting with (?> as in this 1764 example: 1765 </para> 1766 1767 <programlisting> 1768 (?>\d+)foo 1769 </programlisting> 1770 1771 <para> 1772 This kind of parenthesis "locks up" the part of the pattern it contains 1773 once it has matched, and a failure further into the pattern is 1774 prevented from backtracking into it. Backtracking past it to previous 1775 items, however, works as normal. 1776 </para> 1777 1778 <para> 1779 An alternative description is that a subpattern of this type matches 1780 the string of characters that an identical standalone pattern would 1781 match, if anchored at the current point in the string. 1782 </para> 1783 1784 <para> 1785 Atomic grouping subpatterns are not capturing subpatterns. Simple cases 1786 such as the above example can be thought of as a maximizing repeat that 1787 must swallow everything it can. So, while both \d+ and \d+? are prepared 1788 to adjust the number of digits they match in order to make the 1789 rest of the pattern match, (?>\d+) can only match an entire sequence of 1790 digits. 1791 </para> 1792 1793 <para> 1794 Atomic groups in general can of course contain arbitrarily complicated 1795 subpatterns, and can be nested. However, when the subpattern for an 1796 atomic group is just a single repeated item, as in the example above, a 1797 simpler notation, called a "possessive quantifier" can be used. This 1798 consists of an additional + character following a quantifier. Using 1799 this notation, the previous example can be rewritten as 1800 </para> 1801 1802 <programlisting> 1803 \d++foo 1804 </programlisting> 1805 1806 <para> 1807 Possessive quantifiers are always greedy; the setting of the 1808 <varname>G_REGEX_UNGREEDY</varname> option is ignored. They are a convenient notation for the 1809 simpler forms of atomic group. However, there is no difference in the 1810 meaning of a possessive quantifier and the equivalent 1811 atomic group, though there may be a performance difference; 1812 possessive quantifiers should be slightly faster. 1813 </para> 1814 1815 <para> 1816 The possessive quantifier syntax is an extension to the Perl syntax. 1817 It was invented by Jeffrey Friedl in the first edition of his book and 1818 then implemented by Mike McCloskey in Sun's Java package. 1819 It ultimately found its way into Perl at release 5.10. 1820 </para> 1821 1822 <para> 1823 GRegex has an optimization that automatically "possessifies" certain simple 1824 pattern constructs. For example, the sequence A+B is treated as A++B because 1825 there is no point in backtracking into a sequence of A's when B must follow. 1826 </para> 1827 1828 <para> 1829 When a pattern contains an unlimited repeat inside a subpattern that 1830 can itself be repeated an unlimited number of times, the use of an 1831 atomic group is the only way to avoid some failing matches taking a 1832 very long time indeed. The pattern 1833 </para> 1834 1835 <programlisting> 1836 (\D+|<\d+>)*[!?] 1837 </programlisting> 1838 1839 <para> 1840 matches an unlimited number of substrings that either consist of non- 1841 digits, or digits enclosed in <>, followed by either ! or ?. When it 1842 matches, it runs quickly. However, if it is applied to 1843 </para> 1844 1845 <programlisting> 1846 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 1847 </programlisting> 1848 1849 <para> 1850 it takes a long time before reporting failure. This is because the 1851 string can be divided between the internal \D+ repeat and the external 1852 * repeat in a large number of ways, and all have to be tried. (The 1853 example uses [!?] rather than a single character at the end, because 1854 GRegex has an optimization that allows for fast failure 1855 when a single character is used. It remember the last single character 1856 that is required for a match, and fail early if it is not present 1857 in the string.) If the pattern is changed so that it uses an atomic 1858 group, like this: 1859 </para> 1860 1861 <programlisting> 1862 ((?>\D+)|<\d+>)*[!?] 1863 </programlisting> 1864 1865 <para> 1866 sequences of non-digits cannot be broken, and failure happens quickly. 1867 </para> 1868 </refsect1> 1869 1870 <refsect1> 1871 <title>Back references</title> 1872 <para> 1873 Outside a character class, a backslash followed by a digit greater than 1874 0 (and possibly further digits) is a back reference to a capturing subpattern 1875 earlier (that is, to its left) in the pattern, provided there have been that 1876 many previous capturing left parentheses. 1877 </para> 1878 1879 <para> 1880 However, if the decimal number following the backslash is less than 10, 1881 it is always taken as a back reference, and causes an error only if 1882 there are not that many capturing left parentheses in the entire pattern. 1883 In other words, the parentheses that are referenced need not be 1884 to the left of the reference for numbers less than 10. A "forward back 1885 reference" of this type can make sense when a repetition is involved and 1886 the subpattern to the right has participated in an earlier iteration. 1887 </para> 1888 1889 <para> 1890 It is not possible to have a numerical "forward back reference" to subpattern 1891 whose number is 10 or more using this syntax because a sequence such as \e50 is 1892 interpreted as a character defined in octal. See the subsection entitled 1893 "Non-printing characters" above for further details of the handling of digits 1894 following a backslash. There is no such problem when named parentheses are used. 1895 A back reference to any subpattern is possible using named parentheses (see below). 1896 </para> 1897 1898 <para> 1899 Another way of avoiding the ambiguity inherent in the use of digits following a 1900 backslash is to use the \g escape sequence (introduced in Perl 5.10.) 1901 This escape must be followed by a positive or a negative number, 1902 optionally enclosed in braces. 1903 </para> 1904 1905 <para> 1906 A positive number specifies an absolute reference without the ambiguity that is 1907 present in the older syntax. It is also useful when literal digits follow the 1908 reference. A negative number is a relative reference. Consider "(abc(def)ghi)\g{-1}", 1909 the sequence \g{-1} is a reference to the most recently started capturing 1910 subpattern before \g, that is, is it equivalent to \2. Similarly, \g{-2} 1911 would be equivalent to \1. The use of relative references can be helpful in 1912 long patterns, and also in patterns that are created by joining together 1913 fragments that contain references within themselves. 1914 </para> 1915 1916 <para> 1917 A back reference matches whatever actually matched the capturing subpattern 1918 in the current string, rather than anything matching 1919 the subpattern itself (see "Subpatterns as subroutines" below for a way 1920 of doing that). So the pattern 1921 </para> 1922 1923 <programlisting> 1924 (sens|respons)e and \1ibility 1925 </programlisting> 1926 1927 <para> 1928 matches "sense and sensibility" and "response and responsibility", but 1929 not "sense and responsibility". If caseful matching is in force at the 1930 time of the back reference, the case of letters is relevant. For example, 1931 </para> 1932 1933 <programlisting> 1934 ((?i)rah)\s+\1 1935 </programlisting> 1936 1937 <para> 1938 matches "rah rah" and "RAH RAH", but not "RAH rah", even though the 1939 original capturing subpattern is matched caselessly. 1940 </para> 1941 1942 <para> 1943 Back references to named subpatterns use the Perl syntax \k<name> or \k'name' 1944 or the Python syntax (?P=name). We could rewrite the above example in either of 1945 the following ways: 1946 </para> 1947 1948 <programlisting> 1949 (?<p1>(?i)rah)\s+\k<p1> 1950 (?P<p1>(?i)rah)\s+(?P=p1) 1951 </programlisting> 1952 1953 <para> 1954 A subpattern that is referenced by name may appear in the pattern before or 1955 after the reference. 1956 </para> 1957 1958 <para> 1959 There may be more than one back reference to the same subpattern. If a 1960 subpattern has not actually been used in a particular match, any back 1961 references to it always fail. For example, the pattern 1962 </para> 1963 1964 <programlisting> 1965 (a|(bc))\2 1966 </programlisting> 1967 1968 <para> 1969 always fails if it starts to match "a" rather than "bc". Because there 1970 may be many capturing parentheses in a pattern, all digits following 1971 the backslash are taken as part of a potential back reference number. 1972 If the pattern continues with a digit character, some delimiter must be 1973 used to terminate the back reference. If the <varname>G_REGEX_EXTENDED</varname> flag is 1974 set, this can be whitespace. Otherwise an empty comment (see "Comments" below) can be used. 1975 </para> 1976 1977 <para> 1978 A back reference that occurs inside the parentheses to which it refers 1979 fails when the subpattern is first used, so, for example, (a\1) never 1980 matches. However, such references can be useful inside repeated subpatterns. 1981 For example, the pattern 1982 </para> 1983 1984 <programlisting> 1985 (a|b\1)+ 1986 </programlisting> 1987 1988 <para> 1989 matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration 1990 of the subpattern, the back reference matches the character 1991 string corresponding to the previous iteration. In order for this to 1992 work, the pattern must be such that the first iteration does not need 1993 to match the back reference. This can be done using alternation, as in 1994 the example above, or by a quantifier with a minimum of zero. 1995 </para> 1996 </refsect1> 1997 1998 <refsect1> 1999 <title>Assertions</title> 2000 <para> 2001 An assertion is a test on the characters following or preceding the 2002 current matching point that does not actually consume any characters. 2003 The simple assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are 2004 described above. 2005 </para> 2006 2007 <para> 2008 More complicated assertions are coded as subpatterns. There are two 2009 kinds: those that look ahead of the current position in the 2010 string, and those that look behind it. An assertion subpattern is 2011 matched in the normal way, except that it does not cause the current 2012 matching position to be changed. 2013 </para> 2014 2015 <para> 2016 Assertion subpatterns are not capturing subpatterns, and may not be 2017 repeated, because it makes no sense to assert the same thing several 2018 times. If any kind of assertion contains capturing subpatterns within 2019 it, these are counted for the purposes of numbering the capturing 2020 subpatterns in the whole pattern. However, substring capturing is carried 2021 out only for positive assertions, because it does not make sense for 2022 negative assertions. 2023 </para> 2024 2025 <refsect2> 2026 <title>Lookahead assertions</title> 2027 <para> 2028 Lookahead assertions start with (?= for positive assertions and (?! for 2029 negative assertions. For example, 2030 </para> 2031 2032 <programlisting> 2033 \w+(?=;) 2034 </programlisting> 2035 2036 <para> 2037 matches a word followed by a semicolon, but does not include the semicolon 2038 in the match, and 2039 </para> 2040 2041 <programlisting> 2042 foo(?!bar) 2043 </programlisting> 2044 2045 <para> 2046 matches any occurrence of "foo" that is not followed by "bar". Note 2047 that the apparently similar pattern 2048 </para> 2049 2050 <programlisting> 2051 (?!foo)bar 2052 </programlisting> 2053 2054 <para> 2055 does not find an occurrence of "bar" that is preceded by something 2056 other than "foo"; it finds any occurrence of "bar" whatsoever, because 2057 the assertion (?!foo) is always true when the next three characters are 2058 "bar". A lookbehind assertion is needed to achieve the other effect. 2059 </para> 2060 2061 <para> 2062 If you want to force a matching failure at some point in a pattern, the 2063 most convenient way to do it is with (?!) because an empty string 2064 always matches, so an assertion that requires there not to be an empty 2065 string must always fail. 2066 </para> 2067 </refsect2> 2068 2069 <refsect2> 2070 <title>Lookbehind assertions</title> 2071 <para> 2072 Lookbehind assertions start with (?<= for positive assertions and (?<! 2073 for negative assertions. For example, 2074 </para> 2075 2076 <programlisting> 2077 (?<!foo)bar 2078 </programlisting> 2079 2080 <para> 2081 does find an occurrence of "bar" that is not preceded by "foo". The 2082 contents of a lookbehind assertion are restricted such that all the 2083 strings it matches must have a fixed length. However, if there are 2084 several top-level alternatives, they do not all have to have the same 2085 fixed length. Thus 2086 </para> 2087 2088 <programlisting> 2089 (?<=bullock|donkey) 2090 </programlisting> 2091 2092 <para> 2093 is permitted, but 2094 </para> 2095 2096 <programlisting> 2097 (?<!dogs?|cats?) 2098 </programlisting> 2099 2100 <para> 2101 causes an error at compile time. Branches that match different length 2102 strings are permitted only at the top level of a lookbehind assertion. 2103 An assertion such as 2104 </para> 2105 2106 <programlisting> 2107 (?<=ab(c|de)) 2108 </programlisting> 2109 2110 <para> 2111 is not permitted, because its single top-level branch can match two 2112 different lengths, but it is acceptable if rewritten to use two top- 2113 level branches: 2114 </para> 2115 2116 <programlisting> 2117 (?<=abc|abde) 2118 </programlisting> 2119 2120 <para> 2121 The implementation of lookbehind assertions is, for each alternative, 2122 to temporarily move the current position back by the fixed length and 2123 then try to match. If there are insufficient characters before the 2124 current position, the assertion fails. 2125 </para> 2126 2127 <para> 2128 GRegex does not allow the \C escape (which matches a single byte in UTF-8 2129 mode) to appear in lookbehind assertions, because it makes it impossible 2130 to calculate the length of the lookbehind. The \X and \R escapes, which can 2131 match different numbers of bytes, are also not permitted. 2132 </para> 2133 2134 <para> 2135 Possessive quantifiers can be used in conjunction with lookbehind assertions to 2136 specify efficient matching at the end of the subject string. Consider a simple 2137 pattern such as 2138 </para> 2139 2140 <programlisting> 2141 abcd$ 2142 </programlisting> 2143 2144 <para> 2145 when applied to a long string that does not match. Because matching 2146 proceeds from left to right, GRegex will look for each "a" in the string 2147 and then see if what follows matches the rest of the pattern. If the 2148 pattern is specified as 2149 </para> 2150 2151 <programlisting> 2152 ^.*abcd$ 2153 </programlisting> 2154 2155 <para> 2156 the initial .* matches the entire string at first, but when this fails 2157 (because there is no following "a"), it backtracks to match all but the 2158 last character, then all but the last two characters, and so on. Once 2159 again the search for "a" covers the entire string, from right to left, 2160 so we are no better off. However, if the pattern is written as 2161 </para> 2162 2163 <programlisting> 2164 ^.*+(?<=abcd) 2165 </programlisting> 2166 2167 <para> 2168 there can be no backtracking for the .*+ item; it can match only the 2169 entire string. The subsequent lookbehind assertion does a single test 2170 on the last four characters. If it fails, the match fails immediately. 2171 For long strings, this approach makes a significant difference to the 2172 processing time. 2173 </para> 2174 </refsect2> 2175 2176 <refsect2> 2177 <title>Using multiple assertions</title> 2178 <para> 2179 Several assertions (of any sort) may occur in succession. For example, 2180 </para> 2181 2182 <programlisting> 2183 (?<=\d{3})(?<!999)foo 2184 </programlisting> 2185 2186 <para> 2187 matches "foo" preceded by three digits that are not "999". Notice that 2188 each of the assertions is applied independently at the same point in 2189 the string. First there is a check that the previous three 2190 characters are all digits, and then there is a check that the same 2191 three characters are not "999". This pattern does not match "foo" preceded 2192 by six characters, the first of which are digits and the last 2193 three of which are not "999". For example, it doesnt match "123abcfoo". 2194 A pattern to do that is 2195 </para> 2196 2197 <programlisting> 2198 (?<=\d{3}...)(?<!999)foo 2199 </programlisting> 2200 2201 <para> 2202 This time the first assertion looks at the preceding six characters, 2203 checking that the first three are digits, and then the second assertion 2204 checks that the preceding three characters are not "999". 2205 </para> 2206 2207 <para> 2208 Assertions can be nested in any combination. For example, 2209 </para> 2210 2211 <programlisting> 2212 (?<=(?<!foo)bar)baz 2213 </programlisting> 2214 2215 <para> 2216 matches an occurrence of "baz" that is preceded by "bar" which in turn 2217 is not preceded by "foo", while 2218 </para> 2219 2220 <programlisting> 2221 (?<=\d{3}(?!999)...)foo 2222 </programlisting> 2223 2224 <para> 2225 is another pattern that matches "foo" preceded by three digits and any 2226 three characters that are not "999". 2227 </para> 2228 </refsect2> 2229 </refsect1> 2230 2231 <refsect1> 2232 <title>Conditional subpatterns</title> 2233 <para> 2234 It is possible to cause the matching process to obey a subpattern 2235 conditionally or to choose between two alternative subpatterns, depending 2236 on the result of an assertion, or whether a previous capturing subpattern 2237 matched or not. The two possible forms of conditional subpattern are 2238 </para> 2239 2240 <programlisting> 2241 (?(condition)yes-pattern) 2242 (?(condition)yes-pattern|no-pattern) 2243 </programlisting> 2244 2245 <para> 2246 If the condition is satisfied, the yes-pattern is used; otherwise the 2247 no-pattern (if present) is used. If there are more than two alternatives 2248 in the subpattern, a compile-time error occurs. 2249 </para> 2250 2251 <para> 2252 There are four kinds of condition: references to subpatterns, references to 2253 recursion, a pseudo-condition called DEFINE, and assertions. 2254 </para> 2255 2256 <refsect2> 2257 <title>Checking for a used subpattern by number</title> 2258 <para> 2259 If the text between the parentheses consists of a sequence of digits, the 2260 condition is true if the capturing subpattern of that number has previously 2261 matched. 2262 </para> 2263 2264 <para> 2265 Consider the following pattern, which contains non-significant white space 2266 to make it more readable (assume the <varname>G_REGEX_EXTENDED</varname>) 2267 and to divide it into three parts for ease of discussion: 2268 </para> 2269 2270 <programlisting> 2271 ( \( )? [^()]+ (?(1) \) ) 2272 </programlisting> 2273 2274 <para> 2275 The first part matches an optional opening parenthesis, and if that 2276 character is present, sets it as the first captured substring. The second 2277 part matches one or more characters that are not parentheses. The 2278 third part is a conditional subpattern that tests whether the first set 2279 of parentheses matched or not. If they did, that is, if string started 2280 with an opening parenthesis, the condition is true, and so the yes-pattern 2281 is executed and a closing parenthesis is required. Otherwise, 2282 since no-pattern is not present, the subpattern matches nothing. In 2283 other words, this pattern matches a sequence of non-parentheses, 2284 optionally enclosed in parentheses. 2285 </para> 2286 </refsect2> 2287 2288 <refsect2> 2289 <title>Checking for a used subpattern by name</title> 2290 <para> 2291 Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used 2292 subpattern by name, the Python syntax (?(name)...) is also recognized. However, 2293 there is a possible ambiguity with this syntax, because subpattern names may 2294 consist entirely of digits. GRegex looks first for a named subpattern; if it 2295 cannot find one and the name consists entirely of digits, GRegex looks for a 2296 subpattern of that number, which must be greater than zero. Using subpattern 2297 names that consist entirely of digits is not recommended. 2298 </para> 2299 2300 <para> 2301 Rewriting the above example to use a named subpattern gives this: 2302 </para> 2303 2304 <programlisting> 2305 (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) 2306 </programlisting> 2307 </refsect2> 2308 2309 <refsect2> 2310 <title>Checking for pattern recursion</title> 2311 <para> 2312 If the condition is the string (R), and there is no subpattern with the name R, 2313 the condition is true if a recursive call to the whole pattern or any 2314 subpattern has been made. If digits or a name preceded by ampersand follow the 2315 letter R, for example: 2316 </para> 2317 2318 <programlisting> 2319 (?(R3)...) 2320 (?(R&name)...) 2321 </programlisting> 2322 2323 <para> 2324 the condition is true if the most recent recursion is into the subpattern whose 2325 number or name is given. This condition does not check the entire recursion 2326 stack. 2327 </para> 2328 2329 <para> 2330 At "top level", all these recursion test conditions are false. Recursive 2331 patterns are described below. 2332 </para> 2333 </refsect2> 2334 2335 <refsect2> 2336 <title>Defining subpatterns for use by reference only</title> 2337 <para> 2338 If the condition is the string (DEFINE), and there is no subpattern with the 2339 name DEFINE, the condition is always false. In this case, there may be only one 2340 alternative in the subpattern. It is always skipped if control reaches this 2341 point in the pattern; the idea of DEFINE is that it can be used to define 2342 "subroutines" that can be referenced from elsewhere. (The use of "subroutines" 2343 is described below.) For example, a pattern to match an IPv4 address could be 2344 written like this (ignore whitespace and line breaks): 2345 </para> 2346 2347 <programlisting> 2348 (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) 2349 \b (?&byte) (\.(?&byte)){3} \b 2350 </programlisting> 2351 2352 <para> 2353 The first part of the pattern is a DEFINE group inside which a another group 2354 named "byte" is defined. This matches an individual component of an IPv4 2355 address (a number less than 256). When matching takes place, this part of the 2356 pattern is skipped because DEFINE acts like a false condition. 2357 </para> 2358 2359 <para> 2360 The rest of the pattern uses references to the named group to match the four 2361 dot-separated components of an IPv4 address, insisting on a word boundary at 2362 each end. 2363 </para> 2364 </refsect2> 2365 2366 <refsect2> 2367 <title>Assertion conditions</title> 2368 <para> 2369 If the condition is not in any of the above formats, it must be an 2370 assertion. This may be a positive or negative lookahead or lookbehind 2371 assertion. Consider this pattern, again containing non-significant 2372 white space, and with the two alternatives on the second line: 2373 </para> 2374 2375 <programlisting> 2376 (?(?=[^a-z]*[a-z]) 2377 \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) 2378 </programlisting> 2379 2380 <para> 2381 The condition is a positive lookahead assertion that matches an 2382 optional sequence of non-letters followed by a letter. In other words, 2383 it tests for the presence of at least one letter in the string. If a 2384 letter is found, the string is matched against the first alternative; 2385 otherwise it is matched against the second. This pattern matches 2386 strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are 2387 letters and dd are digits. 2388 </para> 2389 </refsect2> 2390 </refsect1> 2391 2392 <refsect1> 2393 <title>Comments</title> 2394 <para> 2395 The sequence (?# marks the start of a comment that continues up to the 2396 next closing parenthesis. Nested parentheses are not permitted. The 2397 characters that make up a comment play no part in the pattern matching 2398 at all. 2399 </para> 2400 2401 <para> 2402 If the <varname>G_REGEX_EXTENDED</varname> option is set, an unescaped # 2403 character outside a character class introduces a comment that continues to 2404 immediately after the next newline in the pattern. 2405 </para> 2406 </refsect1> 2407 2408 <refsect1> 2409 <title>Recursive patterns</title> 2410 <para> 2411 Consider the problem of matching a string in parentheses, allowing for 2412 unlimited nested parentheses. Without the use of recursion, the best 2413 that can be done is to use a pattern that matches up to some fixed 2414 depth of nesting. It is not possible to handle an arbitrary nesting 2415 depth. 2416 </para> 2417 2418 <para> 2419 For some time, Perl has provided a facility that allows regular expressions to 2420 recurse (amongst other things). It does this by interpolating Perl code in the 2421 expression at run time, and the code can refer to the expression itself. A Perl 2422 pattern using code interpolation to solve the parentheses problem can be 2423 created like this: 2424 </para> 2425 2426 <programlisting> 2427 $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; 2428 </programlisting> 2429 2430 <para> 2431 The (?p{...}) item interpolates Perl code at run time, and in this case refers 2432 recursively to the pattern in which it appears. 2433 </para> 2434 2435 <para> 2436 Obviously, GRegex cannot support the interpolation of Perl code. Instead, it 2437 supports special syntax for recursion of the entire pattern, and also for 2438 individual subpattern recursion. This kind of recursion was introduced into 2439 Perl at release 5.10. 2440 </para> 2441 2442 <para> 2443 A special item that consists of (? followed by a number greater than zero and a 2444 closing parenthesis is a recursive call of the subpattern of the given number, 2445 provided that it occurs inside that subpattern. (If not, it is a "subroutine" 2446 call, which is described in the next section.) The special item (?R) or (?0) is 2447 a recursive call of the entire regular expression. 2448 </para> 2449 2450 <para> 2451 In GRegex (like Python, but unlike Perl), a recursive subpattern call is always 2452 treated as an atomic group. That is, once it has matched some of the subject 2453 string, it is never re-entered, even if it contains untried alternatives and 2454 there is a subsequent matching failure. 2455 </para> 2456 2457 <para> 2458 This pattern solves the nested parentheses problem (assume the 2459 <varname>G_REGEX_EXTENDED</varname> option is set so that white space is 2460 ignored): 2461 </para> 2462 2463 <programlisting> 2464 \( ( (?>[^()]+) | (?R) )* \) 2465 </programlisting> 2466 2467 <para> 2468 First it matches an opening parenthesis. Then it matches any number of 2469 substrings which can either be a sequence of non-parentheses, or a 2470 recursive match of the pattern itself (that is, a correctly parenthesized 2471 substring). Finally there is a closing parenthesis. 2472 </para> 2473 2474 <para> 2475 If this were part of a larger pattern, you would not want to recurse 2476 the entire pattern, so instead you could use this: 2477 </para> 2478 2479 <programlisting> 2480 ( \( ( (?>[^()]+) | (?1) )* \) ) 2481 </programlisting> 2482 2483 <para> 2484 We have put the pattern into parentheses, and caused the recursion to 2485 refer to them instead of the whole pattern. In a larger pattern, keeping 2486 track of parenthesis numbers can be tricky. It may be more convenient to 2487 use named parentheses instead. 2488 The Perl syntax for this is (?&name); GRegex also supports the(?P>name) 2489 syntac. We could rewrite the above example as follows: 2490 </para> 2491 2492 <programlisting> 2493 (?<pn> \( ( (?>[^()]+) | (?&pn) )* \) ) 2494 </programlisting> 2495 2496 <para> 2497 If there is more than one subpattern with the same name, the earliest one is 2498 used. This particular example pattern contains nested unlimited repeats, and so 2499 the use of atomic grouping for matching strings of non-parentheses is important 2500 when applying the pattern to strings that do not match. 2501 For example, when this pattern is applied to 2502 </para> 2503 2504 <programlisting> 2505 (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() 2506 </programlisting> 2507 2508 <para> 2509 it yields "no match" quickly. However, if atomic grouping is not used, 2510 the match runs for a very long time indeed because there are so many 2511 different ways the + and * repeats can carve up the string, and all 2512 have to be tested before failure can be reported. 2513 </para> 2514 2515 <para> 2516 At the end of a match, the values set for any capturing subpatterns are 2517 those from the outermost level of the recursion at which the subpattern 2518 value is set. 2519 2520 <!-- Callouts are not supported by GRegex 2521 If you want to obtain intermediate values, a callout 2522 function can be used (see below and the pcrecallout documentation). --> 2523 2524 If the pattern above is matched against 2525 </para> 2526 2527 <programlisting> 2528 (ab(cd)ef) 2529 </programlisting> 2530 2531 <para> 2532 the value for the capturing parentheses is "ef", which is the last 2533 value taken on at the top level. If additional parentheses are added, 2534 giving 2535 </para> 2536 2537 <programlisting> 2538 \( ( ( (?>[^()]+) | (?R) )* ) \) 2539 ^ ^ 2540 ^ ^ 2541 </programlisting> 2542 2543 <para> 2544 the string they capture is "ab(cd)ef", the contents of the top level 2545 parentheses. 2546 </para> 2547 2548 <para> 2549 Do not confuse the (?R) item with the condition (R), which tests for 2550 recursion. Consider this pattern, which matches text in angle brackets, 2551 allowing for arbitrary nesting. Only digits are allowed in nested 2552 brackets (that is, when recursing), whereas any characters are permitted 2553 at the outer level. 2554 </para> 2555 2556 <programlisting> 2557 < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > 2558 </programlisting> 2559 2560 <para> 2561 In this pattern, (?(R) is the start of a conditional subpattern, with 2562 two different alternatives for the recursive and non-recursive cases. 2563 The (?R) item is the actual recursive call. 2564 </para> 2565 </refsect1> 2566 2567 <refsect1> 2568 <title>Subpatterns as subroutines</title> 2569 <para> 2570 If the syntax for a recursive subpattern reference (either by number or 2571 by name) is used outside the parentheses to which it refers, it operates 2572 like a subroutine in a programming language. The "called" subpattern may 2573 be defined before or after the reference. An earlier example pointed out 2574 that the pattern 2575 </para> 2576 2577 <programlisting> 2578 (sens|respons)e and \1ibility 2579 </programlisting> 2580 2581 <para> 2582 matches "sense and sensibility" and "response and responsibility", but 2583 not "sense and responsibility". If instead the pattern 2584 </para> 2585 2586 <programlisting> 2587 (sens|respons)e and (?1)ibility 2588 </programlisting> 2589 2590 <para> 2591 is used, it does match "sense and responsibility" as well as the other 2592 two strings. Another example is given in the discussion of DEFINE above. 2593 </para> 2594 2595 <para> 2596 Like recursive subpatterns, a "subroutine" call is always treated as an atomic 2597 group. That is, once it has matched some of the string, it is never 2598 re-entered, even if it contains untried alternatives and there is a subsequent 2599 matching failure. 2600 </para> 2601 2602 <para> 2603 When a subpattern is used as a subroutine, processing options such as 2604 case-independence are fixed when the subpattern is defined. They cannot be 2605 changed for different calls. For example, consider this pattern: 2606 </para> 2607 2608 <programlisting> 2609 (abc)(?i:(?1)) 2610 </programlisting> 2611 2612 <para> 2613 It matches "abcabc". It does not match "abcABC" because the change of 2614 processing option does not affect the called subpattern. 2615 </para> 2616 </refsect1> 2617 2618 <!-- Callouts are not supported by GRegex 2619 <refsect1> 2620 <title>Callouts</title> 2621 <para> 2622 Perl has a feature whereby using the sequence (?{...}) causes arbitrary 2623 Perl code to be obeyed in the middle of matching a regular expression. 2624 This makes it possible, amongst other things, to extract different substrings that match the same pair of parentheses when there is a repetition. 2625 </para> 2626 2627 <para> 2628 PCRE provides a similar feature, but of course it cannot obey arbitrary 2629 Perl code. The feature is called "callout". The caller of PCRE provides 2630 an external function by putting its entry point in the global variable 2631 pcre_callout. By default, this variable contains NULL, which disables 2632 all calling out. 2633 </para> 2634 2635 <para> 2636 Within a regular expression, (?C) indicates the points at which the 2637 external function is to be called. If you want to identify different 2638 callout points, you can put a number less than 256 after the letter C. 2639 The default value is zero. For example, this pattern has two callout 2640 points: 2641 </para> 2642 2643 <programlisting> 2644 (?C1)abc(?C2)def 2645 </programlisting> 2646 2647 <para> 2648 If the PCRE_AUTO_CALLOUT flag is passed to pcre_compile(), callouts are 2649 automatically installed before each item in the pattern. They are all 2650 numbered 255. 2651 </para> 2652 2653 <para> 2654 During matching, when PCRE reaches a callout point (and pcre_callout is 2655 set), the external function is called. It is provided with the number 2656 of the callout, the position in the pattern, and, optionally, one item 2657 of data originally supplied by the caller of pcre_exec(). The callout 2658 function may cause matching to proceed, to backtrack, or to fail altogether. A complete description of the interface to the callout function 2659 is given in the pcrecallout documentation. 2660 </para> 2661 </refsect1> 2662 --> 2663 2664 <refsect1> 2665 <title>Copyright</title> 2666 <para> 2667 This document was copied and adapted from the PCRE documentation, 2668 specifically from the man page for pcrepattern. 2669 The original copyright note is: 2670 </para> 2671 2672 <programlisting> 2673 Copyright (c) 1997-2006 University of Cambridge. 2674 2675 Redistribution and use in source and binary forms, with or without 2676 modification, are permitted provided that the following conditions are met: 2677 2678 * Redistributions of source code must retain the above copyright notice, 2679 this list of conditions and the following disclaimer. 2680 2681 * Redistributions in binary form must reproduce the above copyright 2682 notice, this list of conditions and the following disclaimer in the 2683 documentation and/or other materials provided with the distribution. 2684 2685 * Neither the name of the University of Cambridge nor the name of Google 2686 Inc. nor the names of their contributors may be used to endorse or 2687 promote products derived from this software without specific prior 2688 written permission. 2689 2690 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 2691 AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 2692 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 2693 ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 2694 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 2695 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 2696 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 2697 INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 2698 CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 2699 ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 2700 POSSIBILITY OF SUCH DAMAGE. 2701 </programlisting> 2702 </refsect1> 2703 2704 </refentry> 2705