c# regular expressions c#.net software development version 1.0

C# Regular Expressions

C# .Net Software Development

Version 1.0

Copyright © 2007-2011 by Dennis A. Fairclough all rights reserved 2

Introducing Regular Expressions

String pattern matching tool

Regular expressions constitute a language• C# regular expressions are classes in the

language

Used in many programming languages

CNS 3240: Computational Theory• Discusses the theory of Regular Expressions


Pattern Matching Match any single character in the bracketed set (range)

• [a-zA-Z]

Anything not in brackets (range) is matched exactly (spaces)• Except for special characters (esc with \ )• abc[a-zA-Z]

* Match the preceding pattern (range) zero or more times (Kleen Star)• [a-zA-Z]*

+ Match the preceding pattern (range) one or more times• [a-zA-Z]+


Language Elements ( ) pattern group | “or”, choose between patterns [ ] defines a range of characters [A-Z]upper case character range { } used as a quantifier \ escape character . matches any character ^ beginning of line $ end of line [^] not character range


Quantifiers * Matches zero or more + Matches one or more ? Matches zero or one {n} Matches exactly n {n,} Matches at least n {n,m} Matches at least n, up to m

Greedy quantifiers always span the largest pattern they can match

Lazy quantifiers always take the smallest pattern they can match• The lazy quantifiers “?”

Beware of spaces!


Character Classes \w Matches any word character

• Same as: [a-zA-Z_0-9] \W Matches any non-word character

• Same as: [^a-zA-Z_0-9] \s Matches any white-space character

• Same as: [ \f\n\r\t\v] \S Matches any non-white-space character

• Same as: [^ \f\n\r\t\v]

\d Matches any digit character• Same as: [0-9]

\D Matches any non-digit character• Same as: [^0-9]

space

Character

\*

\+

\?



\|

\\

\!

\.

\[

\]

\^

\{

\}

\$

\#

Escaping CharactersOrdinary characters, other than the ones in the table below, match themselves. That is, an "a" in a regular expression specifies that input should match an "a“.

7Copyright © 2007-2011 by Dennis A. Fairclough all rights reserved

Regular expressions have the following metacharacters, which have a special rather than literal meaning:

\ * + ? | { [ ( ) ^ $ . #


^ has two context-dependent meanings: an anchor and a character class negator.

$ has two context-dependent meanings: an anchor and a replacement group denoter.

AnchorsThe anchors ^ and $ match a particular position. By default:

^Matches the start of the string.

$Matches the end of the string.

Escaped Character Description ordinary characters Characters other than . $ ^ { [ ( | ) * + ? \ match themselves. \a Matches a bell (alarm) \u0007. \b Matches a backspace \u0008 if in a [] character class; otherwise, see the note following this table. \t Matches a tab \u0009. \r Matches a carriage return \u000D. \v Matches a vertical tab \u000B. \f Matches a form feed \u000C. \n Matches a new line \u000A. \e Matches an escape \u001B. \040 Matches an ASCII character as octal (up to three digits); numbers with no leading zero are backreferences if they have only one digit or if they correspond to a capturing group number. (For more information, see Backreferences.) For example, the character \040 represents a space. \x20 Matches an ASCII character using hexadecimal representation (exactly two digits). \cC Matches an ASCII control character; for example, \cC is control-C. \u0020 Matches a Unicode character using hexadecimal representation (exactly four digits). \ When followed by a character that is not recognized as an escaped character, matches that character. For example, \* is the same as \x2A.Note The escaped character \b is a special case. In a regular expression, \b denotes a word boundary (between \w and \W characters) except within a [] character class, where \b refers to the backspace character. In a replacement pattern, \b always denotes a backspace.


http://msdn.microsoft.com/library/en-us/cpgenref/html/cpconcharacterclasses.asp

http://msdn.microsoft.com/library/en-us/cpguide/html/cpconBackreferences.asp

Character Description of Substitution$number Substitutes the last substring matched by group number number (decimal). ${name} Substitutes the last substring matched by a (?<name> ) group. $$ Substitutes a single "$" literal. $& Substitutes a copy of the entire match itself. $` Substitutes all the text of the input string before the match. $' Substitutes all the text of the input string after the match. $+ Substitutes the last group captured. $_ Substitutes the entire input string.


Character class Description. Matches any character except \n. If modified by the Singleline option, a period character

matches any character. For more information, see Regular Expression Options.

[aeiou] Matches any single character included in the specified set of characters.

[^aeiou] Matches any single character not in the specified set of characters.

[0-9a-fA-F] Use of a hyphen (–) allows specification of contiguous character ranges.

\p{name} Matches any character in the named character class specified by {name}. Supported names are Unicode groups and block ranges. For example, Ll, Nd, Z, IsGreek, IsBoxDrawing.

\P{name} Matches text not included in groups and block ranges specified in {name}.

\w Matches any word character. Equivalent to the Unicode character categories[\p{Ll}\p{Lu}\p{Lt}\p{Lo}\p{Nd}\p{Pc}]. If ECMAScript-compliant behavior is specified with the ECMAScript option, \w is equivalent to [a-zA-Z_0-9].

\W Matches any nonword character. Equivalent to the Unicode categories [^\p{Ll}\p{Lu}\p{Lt}\p{Lo}\p{Nd}\p{Pc}]. If ECMAScript-compliant behavior is specified with the ECMAScript option, \W is equivalent to [^a-zA-Z_0-9].

\s Matches any white-space character. Equivalent to the Unicode character categories [\f\n\r\t\v\x85\p{Z}]. If ECMAScript-compliant behavior is specified with the ECMAScript option, \s is equivalent to [ \f\n\r\t\v].

\S Matches any non-white-space character. Equivalent to the Unicode character categories [^\f\n\r\t\v\x85\p{Z}]. If ECMAScript-compliant behavior is specified with the ECMAScript option, \S is equivalent to [^ \f\n\r\t\v].

\d Matches any decimal digit. Equivalent to \p{Nd} for Unicode and [0-9] for non-Unicode, ECMAScript behavior.

\D Matches any nondigit. Equivalent to \P{Nd} for Unicode and [^0-9] for non-Unicode, ECMAScript behavior.


http://msdn.microsoft.com/library/en-us/cpgenref/html/cpconregularexpressionoptions.asp


Assertion Description

^ Specifies that the match must occur at the beginning of the string or the beginning of the line. For more information, see the Multiline option in Regular Expression Options.

$ Specifies that the match must occur at the end of the string, before \n at the end of the string, or at the end of the line. For more information, see the Multiline option in Regular Expression Options.

\A Specifies that the match must occur at the beginning of the string (ignores the Multiline option).

\Z Specifies that the match must occur at the end of the string or before \n at the end of the string (ignores the Multiline option).

\z Specifies that the match must occur at the end of the string (ignores the Multiline option).

\G Specifies that the match must occur at the point where the previous match ended. When used with Match.NextMatch(), this ensures that matches are all contiguous.

\b Specifies that the match must occur on a boundary between \w (alphanumeric) and \W (nonalphanumeric) characters. The match must occur on word boundaries — that is, at the first or last characters in words separated by any nonalphanumeric characters.

\B Specifies that the match must not occur on a \b boundary.12Copyright © 2007-2011 by Dennis A. Fairclough all rights reserved



Quantifier Description* Specifies zero or more matches; for example, \w* or (abc)*. Equivalent to {0,}.

+ Specifies one or more matches; for example, \w+ or (abc)+. Equivalent to {1,}.

? Specifies zero or one matches; for example, \w? or (abc)?. Equivalent to {0,1}.

{n} Specifies exactly n matches; for example, (pizza){2}.

{n,} Specifies at least n matches; for example, (abc){2,}.

{n,m} Specifies at least n, but no more than m, matches.

*? Specifies the first match that consumes as few repeats as possible (equivalent to lazy *).

+? Specifies as few repeats as possible, but at least one (equivalent to lazy +).

?? Specifies zero repeats if possible, or one (lazy ?).

{n}? Equivalent to {n} (lazy {n}).{n,}? Specifies as few repeats as possible, but at least n (lazy {n,}).

{n,m}? Specifies as few repeats as possible between n and m (lazy {n,m}).


Grouping Construct Description

( ) Captures the matched substring (or noncapturing group; for more information, see the ExplicitCapture option in Regular Expression Options). Captures using () are numbered automatically based on the order of the opening parenthesis, starting from one. The first capture, capture element number zero, is the text matched by the whole regular expression pattern.

(?<name> ) Captures the matched substring into a group name or number name. The string used for name must not contain any punctuation and it cannot begin with a number. You can use single quotes instead of angle brackets; for example, (?'name').

(?<name1-name2> ) Balancing group definition. Deletes the definition of the previously defined group name2 and stores in group name1 the interval between the previously defined name2 group and the current group. If no group name2 is defined, the match backtracks. Because deleting the last definition of name2 reveals the previous definition of name2, this construct allows the stack of captures for group name2 to be used as a counter for keeping track of nested constructs such as parentheses. In this construct, name1 is optional. You can use single quotes instead of angle brackets; for example, (?'name1-name2').

(?: ) Noncapturing group.(?imnsx-imnsx: ) Applies or disables the specified options within the subexpression. For example, (?i-s: )

turns on case insensitivity and disables single-line mode. For more information, see Regular Expression Options.

(?= ) Zero-width positive lookahead assertion. Continues match only if the subexpression matches at this position on the right. For example, \w+(?=\d) matches a word followed by a digit, without matching the digit. This construct does not backtrack.

(?! ) Zero-width negative lookahead assertion. Continues match only if the subexpression does not match at this position on the right. For example, \b(?!un)\w+\b matches words that do not begin with un.

(?<= ) Zero-width positive lookbehind assertion. Continues match only if the subexpression matches at this position on the left. For example, (?<=19)99 matches instances of 99 that follow 19. This construct does not backtrack.

(?<! ) Zero-width negative lookbehind assertion. Continues match only if the subexpression does not match at the position on the left.

(?> ) Nonbacktracking subexpression (also known as a "greedy" subexpression). The subexpression is fully matched once, and then does not participate piecemeal in backtracking. (That is, the subexpression matches only strings that would be matched by the subexpression alone.)




Number Name Pattern

0 0 (default name)

((?<One>abc)/d+)?(?<Two>xyz)(.*)

1 1 (default name)

((?<One>abc)/d+)

2 2 (default name)

(.*)

3 One (?<One>abc)

4 Two (?<Two>xyz)


Backreference construct Definition

\number Backreference. For example, (\w)\1 finds doubled word characters.

\k<name> Named backreference. For example, (?<char>\w)\k<char> finds doubled word characters. The expression (?<43>\w)\43 does the same. You can use single quotes instead of angle brackets; for example, \k'char'.


Alternation construct Definition

| Matches any one of the terms separated by the | (vertical bar) character; for example, cat|dog|tiger. The leftmost successful match wins. Beware of spaces!

(?(expression)yes|no) Matches the "yes" part if the expression matches at this point; otherwise, matches the "no" part. The "no" part can be omitted. The expression can be any valid subexpression, but it is turned into a zero-width assertion, so this syntax is equivalent to (?(?=expression)yes|no). Note that if the expression is the name of a named group or a capturing group number, the alternation construct is interpreted as a capture test (described in the next row of this table). To avoid confusion in these cases, you can spell out the inside (?=expression) explicitly.

(?(name)yes|no) Matches the "yes" part if the named capture string has a match; otherwise, matches the "no" part. The "no" part can be omitted. If the given name does not correspond to the name or number of a capturing group used in this expression, the alternation construct is interpreted as an expression test (described in the preceding row of this table). 17Copyright © 2007-2011 by Dennis A. Fairclough all rights reserved

Construct Definition

(?imnsx-imnsx) Sets or disables options such as case insensitivity to be turned on or off in the middle of a pattern. For information on specific options, see Regular Expression Options. Option changes are effective until the end of the enclosing group. See also the information on the grouping construct (?imnsx-imnsx: ), which is a cleaner form.

(?# ) Inline comment inserted within a regular expression. The comment terminates at the first closing parenthesis character.

# [to end of line] X-mode comment. The comment begins at an unescaped # and continues to the end of the line. (Note that the x option or the RegexOptions.IgnorePatternWhitespace enumerated option must be activated for this kind of comment to be recognized.)



http://msdn.microsoft.com/library/en-us/cpgenref/html/cpcongroupingconstructs.asp

\b[A-Za-z0-9._%-]+@[A-Za-z0-9.-]+\.(com|org|biz|net|info)\b Email Address

.*\.txt File extensions

<([A-Z][A-Z0-9]*)\b[^>]*>(.*?)</\1> HTML Tags

\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b IP Addresses or\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b IP Addresses

[-+]?([0-9]*\.[0-9]+) Floating Point Number

(19|20)\d\d[- /.](0[1-9]|1[012])[- /.](0[1-9]|[12][0-9]|3[01]) Dates (19|20)\d\d or

(0[1-9]|1[012])[- /.](0[1-9]|[12][0-9]|3[01])[- /.](19|20)\d\d Dates

[ \t\r\n\f\v] or [\s] White space

//.*$ Single Line Comment/\*.*?\*/ C Style Comment

"[^"\\\r\n]*(\\.[^"\\\r\n]*)*" Single Line String"[^"\\]*(\\.[^"\\]*)*" Multiline String

\b\d+\b Positive Integer [-+]?\b\d+\b Signed Positive Integer((\b[0-9]+)?\.)?[0-9]+\b Integer & Float(\b[0-9]+\.([0-9]+\b)?|\.[0-9]+\b) Matches a Float

\b(first|second|third|etc)\b Matches Reserved Words 19Copyright © 2007-2011 by Dennis A. Fairclough all rights reserved

enum INSCOPE{ // common input scopes IS_DEFAULT = 0, IS_URL = 1, IS_FILE_FULLFILEPATH = 2, IS_FILE_FILENAME = 3, IS_EMAIL_USERNAME = 4, IS_EMAIL_SMTPEMAILADDRESS = 5, IS_LOGINNAME = 6, IS_PERSONALNAME_FULLNAME = 7, IS_PERSONALNAME_PREFIX = 8, IS_PERSONALNAME_GIVENNAME = 9, IS_PERSONALNAME_MIDDLENAME = 10, IS_PERSONALNAME_SURNAME = 11, IS_PERSONALNAME_SUFFIX = 12, IS_ADDRESS_FULLPOSTALADDRESS = 13, IS_ADDRESS_POSTALCODE = 14, IS_ADDRESS_STREET = 15, IS_ADDRESS_STATEORPROVINCE = 16, IS_ADDRESS_CITY = 17, IS_ADDRESS_COUNTRYNAME = 18, IS_ADDRESS_COUNTRYSHORTNAME = 19, IS_CURRENCY_AMOUNTANDSYMBOL = 20, IS_CURRENCY_AMOUNT = 21, IS_DATE_FULLDATE = 22, IS_DATE_MONTH = 23,IS_DATE_DAY = 24, IS_DATE_YEAR = 25, IS_DATE_MONTHNAME = 26,

IS_DATE_DAYNAME = 27,IS_DIGITS = 28, IS_NUMBER = 29, IS_ONECHAR = 30, IS_PASSWORD = 31, IS_TELEPHONE_FULLTELEPHONENUMBER = 32, IS_TELEPHONE_COUNTRYCODE = 33, IS_TELEPHONE_AREACODE = 34, IS_TELEPHONE_LOCALNUMBER = 35, IS_TIME_FULLTIME = 36, IS_TIME_HOUR = 37, IS_TIME_MINORSEC = 38, IS_NUMBER_FULLWIDTH = 39, IS_ALPHANUMERIC_HALFWIDTH = 40, IS_ALPHANUMERIC_FULLWIDTH = 41, IS_CURRENCY_CHINESE = 42, IS_BOPOMOFO = 43, IS_HIRAGANA = 44, IS_KATAKANA_HALFWIDTH = 45, IiS_KATAKANA_FULLWIDTH = 46,.IS_HANJA = 47, IS_PHRASELIST = -1, IS_REGULAREXPRESSION = -2, IS_SRGS = -3, IS_XML = -4, }



Putting It Together Regular Expression for C# identifiers:

• [a-zA-Z_][a-zA-Z0-9_]* (_ _ reserved)

Floating Point Numbers:• (0|([1-9][0-9]*))?\.[0-9]+

C# Hexidecimal numbers• 0[xX][0-9a-fA-F]+


C# RE Classes

System.Text.RegularExpressions• Regex• Match• MatchCollection• Capture• CaptureCollection• Group


Regex Class Static or Instance?

Static:• Regex exposes static methods for doing RE matching quickly• Use only when need to match very few times

Instance:• Holds a Regular Expression as an object• Compiles the expression to make it faster• Use when need to match many times


Class Diagrams

Regex+Match:Match(string)+MatchCollection:Matches(string)

Match : Group+GroupCollection:Groups+Match:NextMatch()

Group : Capture+CaptureCollection:Captures+bool:Success

Capture+string:Value+int:Length+int:Index


Matchesprivate Regex re1 = new Regex(@"(([2-9]\d{2})-)?(\d{3})-(\d{4})");private string input1 = “801-224-6707";

Match:Value (from Capture)Index (from Capture)Length (from Capture)Success (from Group)NextMatch()Captures (from Group)Groups


Linked Matches

Follow links using NextMatch() Last link Success == false

Match1NextMatch()

Success==true

Match2NextMatch()

Success==true

Match3NextMatch()

Success==true

Match4NextMatch()

Success=false

Match m = re2.Match(input2);while(m.Success){ // work with m here m = m.NextMatch();}


Groups

private Regex re1 = new Regex("(([2-9]\d{2})-)?(\d{3})-(\d{4})");

Group:Value (from Capture)Index (from Capture)Length (from Capture)SuccessCaptures

1

2

0

Groups are defined in perens () Captures a matching substring for

future use• Captures in a Capture object

Group 0 represents the entire expression

3 4


Named Groups

(?<name>expression) Non-capturing group

• (?:expression)

(@"(?:(?<areaCode>[2-9]\d{2})-)?(?<prefix>\d{3})-(?<lastFour>\d{4})

(See RegexGroups Demo)


Regex Members Note: Each method has both a static and instance implementation

Options Escape() GetGroupNames() GetGroupNumbers() GetNameFromNumber() GetNumberFromName() IsMatch() Match() Matches() Replace() Split() Unescape()


Regex.Match() Regex.IsMatch(string input, string pattern)

• returns true if input matches pattern at least once

Regex.Match(string input, string pattern)• Returns a Match object• Use Match.Value to get the string value of the match

Regex.Matches(string input, string pattern)• Returns a MatchCollection of all the occurrences of

pattern in input


Regex Group Name Stuff

GetGroupNames()• Returns all the group names in a string[]

GetGroupNumbers()• Returns the group numbers in an int[]

GetNameFromNumber() GetNumberFromName()


Regex.Escape() If you’re not sure what needs to be escaped?

Regex.Escape(string pattern)• Returns a new string with the necessary characters

escaped

Need to undo it? Regex.Unescape(string pattern)

• Returns a new string with all escape characters removed


Regex.Replace

Refer to a group capture in the regex using a $ Replace(string input, string replacement, int count)

string input2 = "aaabbbccc:aaabbbccc:aaabbbccc";Regex re2 = new Regex("(aaa)(bbb)(ccc)");

Console.WriteLine();Console.WriteLine("Replace...");Console.WriteLine(re2.Replace(input2, "$3$2$1", 1));Console.ReadLine();


Tokenizing a string

Tokenizing means splitting into parts• Why not use String.Split() ?

Leaves empty strings sometimes We must know all possible tokens

(delimiters)• Instead of knowing valid characters

(See SearchingStrings Demo)


Tokenizing using IndexOf()

Exactly what we want• No Extra Empty strings• We only need to know what is a valid

character• Not the most understandable code...



Regex.Split()

Splits the string on a Regular Expression Pattern

string input = "one%%two%%%%three%%%four";Console.WriteLine();Console.WriteLine("Split...");Console.WriteLine(string.Join(",", Regex.Split(input, @"[%]+")));Console.ReadLine();



The Bottom Line

Regular expressions are powerful

They are the industry standard for string pattern matching

They are supported in most language libraries

They are supported in many search tools (grep, as well as VS 2005 search)

What did you learn?

??


c# regular expressions c#.net software development version 1.0

Documents

matches theendof

single character

ascii character

quantifierescape character

preceding pattern range

zupper case character

rights reserved8

rights reserved5quantifiers