Perl Regular Expressions

Perl - Regular Expressions

A regular expression (regex) is a sequence of characters that defines a search pattern. Perl’s regular expression capabilities are one of its most powerful features, heavily influenced by tools like grep, sed, and awk, but significantly extended. They are used for pattern matching, text manipulation, and data validation.

The primary way to apply a regular expression is using the binding operators =~ (matches) and !~ (does not match).

Perl offers three main regular expression operators:

• Match: m/PATTERN/modifiers (or simply /PATTERN/modifiers if slashes are used as delimiters)
• Substitute: s/PATTERN/REPLACEMENT/modifiers
• Transliterate: tr/SEARCHLIST/REPLACEMENTLIST/modifiers (or y///)

Forward slashes / are the traditional delimiters, but you can use almost any non-alphanumeric, non-whitespace character (e.g., m!PATTERN!, s{PATTERN}{REPLACEMENT}, tr[SEARCHLIST][REPLACEMENTLIST]). If you use single quotes as delimiters (e.g., m'PATTERN'), no variable interpolation occurs in PATTERN.

The Match Operator (m//)

The match operator m// tests a string against a regular expression. In scalar context, it returns true if the pattern matches and false otherwise. In list context, it returns a list of captured substrings (from parentheses in the pattern).

#!/usr/bin/perl
use strict;
use warnings;

my $text = "Hello Perl World, Perl is powerful.";

if ($text =~ m/Perl/) {
    print "'Perl' found in the text.\n";
} else {
    print "'Perl' not found.\n";
}

# Using different delimiters
if ($text =~ m!powerful!) {
    print "'powerful' found using 'm!!' operator.\n";
}

# Omitting 'm' with / delimiters
if ($text =~ /World/) {
    print "'World' found using '//' operator.\n";
}

Output:

'Perl' found in the text.
'powerful' found using 'm!!' operator.
'World' found using '//' operator.

In list context, m// returns captured groups:

#!/usr/bin/perl
use strict;
use warnings;

my $time_string = "Time: 10:30:45";
my ($hours, $minutes, $seconds) = ($time_string =~ m/(\d+):(\d+):(\d+)/);

if (defined $hours) {
    print "Hours: $hours, Minutes: $minutes, Seconds: $seconds\n";
} else {
    print "Time pattern not found.\n";
}

Output:

Hours: 10, Minutes: 30, Seconds: 45

Match Operator Modifiers

Modifiers alter the behavior of the match operator:

Modifier	Description
i	Case-insensitive matching.
m	Multiline mode: ^ and $ match at the beginning/end of lines within the string, in addition to start/end of string.
s	Single-line mode: . (dot) matches any character, including newline (\n). Without /s, . does not match \n.
x	Extended mode: Allows whitespace and comments (# ...) within the pattern for readability.
g	Global matching: Finds all occurrences. In list context, returns all captures from all matches. In scalar context within a while loop, iterates through matches.
o	Compile once: Compiles the regular expression only once, even if it contains interpolated variables. Less critical in modern Perl versions due to optimizations, but can be useful.
c	Continue search: When used with /g, allows the search to continue after a global match fails at a certain position (prevents resetting pos() on failure).

Named Captures (Modern Perl)

Instead of relying on numbered capture groups like $1, $2, modern Perl (5.10+) allows named captures using the (?<name>PATTERN) syntax. These are accessible via the %+ hash.

#!/usr/bin/perl
use strict;
use warnings;
use v5.10; # Enables named captures and %+ hash

my $data = "Date: 2023-10-26";
if ($data =~ m/Date: (?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})/) {
    print "Year: $ENV{'+'}{'year'}\n"; # Access via %+ hash (older syntax was $^{+}{name})
    print "Month: $ENV{'+'}{'month'}\n";
    print "Day: $ENV{'+'}{'day'}\n";
} else {
    print "Date not found or format incorrect.\n";
}

Output:

Year: 2023
Month: 10
Day: 26

Regular Expression Variables

Perl provides special variables that are set after a successful match:

• $& (or $MATCH with use English;): The entire string matched by the regex.
• $ (or $PREMATCH): The part of the string before the matched portion.
• $' (or $POSTMATCH): The part of the string after the matched portion.
• $1, $2, … : The strings captured by the first, second, … pair of parentheses in the pattern.
• %+: A hash containing named captures (if used).
• %-: A hash containing all captures from groups that have successfully matched, indexed by group name. Useful with (?|...) branch reset grouping.

Note: Using $, $&, or $' can incur a performance penalty on older Perl versions because Perl has to populate them for every match. Modern Perls (5.16+) have optimized this significantly. For Perl 5.10+, if you need these, using ${^PREMATCH}, ${^MATCH}, ${^POSTMATCH} (requires use re 'eval'; or is available under use v5.10) can sometimes be more performant or using the /p modifier (Perl 5.20+) to populate @{-} and @{-}. Generally, using capture groups ($1, $2, or named captures) is preferred for clarity and performance.

#!/usr/bin/perl
use strict;
use warnings;

my $string = "The quick brown fox jumps over the lazy_dog.";
if ($string =~ /(brown fox)/) {
    print "Matched: '$&'\n";
    print "Before: '$`'\n";
    print "After: '$''\n";
    print "Captured (\$1): '$1'\n";
}

Output:

Matched: 'brown fox'
Before: 'The quick '
After: ' jumps over the lazy_dog.'
Captured ($1): 'brown fox'

The Substitution Operator (s///)

The substitution operator s/PATTERN/REPLACEMENT/modifiers searches for PATTERN and replaces it with REPLACEMENT.

#!/usr/bin/perl
use strict;
use warnings;

my $sentence = "I like apples and apples are tasty.";
$sentence =~ s/apples/bananas/;
# Replaces only the first occurrence
print "1. $sentence\n";

$sentence =~ s/bananas/oranges/g;
# Replaces all occurrences (due to /g)
print "2. $sentence\n";

Output:

1. I like bananas and apples are tasty.
2. I like oranges and oranges are tasty.

Substitution Operator Modifiers

Includes i, m, s, x, g, o from match modifiers, plus:

Modifier	Description
e	Evaluates REPLACEMENT as Perl code, and uses its return value as the replacement string. Use ee for two evaluations.
r	Non-destructive substitution: Returns the modified string, leaving the original string unchanged. This is very useful as it avoids modifying the variable in place and allows for easier chaining.

Example with /e and /r:

#!/usr/bin/perl
use strict;
use warnings;

my $text = "Value: 10, Value: 20";
# Using /e to perform calculation
$text =~ s/(\d+)/$1 * 2/eg;
print "Evaluated: $text\n"; # Output: Evaluated: Value: 20, Value: 40

my $original = "hello world";
my $modified = $original =~ s/world/Perl/r;
print "Original: $original\n"; # Output: Original: hello world
print "Modified: $modified\n"; # Output: Modified: hello Perl

The Translation Operator (tr/// or y///)

tr/SEARCHLIST/REPLACEMENTLIST/modifiers transliterates characters from SEARCHLIST to corresponding characters in REPLACEMENTLIST. It does not use regular expression patterns.

#!/usr/bin/perl
use strict;
use warnings;

my $string = "Hello World 123";
$string =~ tr/A-Za-z/a-zA-Z/; # Swap case
print "Case swapped: $string\n";

$string =~ tr/0-9/X/; # Replace all digits with X
print "Digits replaced: $string\n";

Output:

Case swapped: hELLO wORLD 123
Digits replaced: hELLO wORLD XXX

Translation Operator Modifiers

Modifier	Description
c	Complement SEARCHLIST. All characters NOT in SEARCHLIST are targeted.
d	Delete found but unreplaced characters (if REPLACEMENTLIST is shorter than SEARCHLIST).
s	Squash duplicate replaced characters into a single character.

#!/usr/bin/perl
use strict;
use warnings;

my $data = "Perl---Programming---Rocks!!!";
$data =~ tr/-/ /s; # Squash multiple hyphens to a single space
print "Squashed: $data\n";

my $text = "abc123xyz";
$text =~ tr/0-9//d; # Delete all digits
print "Digits deleted: $text\n";

Output:

Squashed: Perl Programming Rocks!!!
Digits deleted: abcxyz

Core Regular Expression Syntax

Perl’s regex syntax is rich. Here’s a summary of common elements:

Pattern	Description
^	Matches beginning of the string (or line, with /m).
$	Matches end of the string (or line, with /m).
.	Matches any single character except newline (unless /s modifier is used).
[…]	Character class: Matches any single character within the brackets (e.g., [aeiou]). Ranges can be used (e.g., [a-z0-9]).
[^…]	Negated character class: Matches any single character NOT within the brackets.
*	Matches 0 or more occurrences of the preceding expression.
+	Matches 1 or more occurrences of the preceding expression.
?	Matches 0 or 1 occurrence of the preceding expression (makes it optional).
{n}	Matches exactly n occurrences.
{n,}	Matches n or more occurrences.
{n,m}	Matches at least n and at most m occurrences.
*?, +?, ??, {n,m}?	Non-greedy (minimal) matching versions of quantifiers.
|	Alternation: Matches either the expression before or after it (e.g., cat|dog).
(…)	Grouping: Groups expressions and captures the matched text. Access via $1, $2, etc., or named captures.
(?:…)	Non-capturing group: Groups expressions but does not capture.
\w	Matches a “word” character (alphanumeric plus ”_”). Equivalent to [A-Za-z0-9_]. Locale-dependent.
\W	Matches a non-word character. Equivalent to [^A-Za-z0-9_]. Locale-dependent.
\s	Matches a whitespace character (space, tab, newline, carriage return, form feed).
\S	Matches a non-whitespace character.
\d	Matches a digit. Equivalent to [0-9].
\D	Matches a non-digit. Equivalent to [^0-9].
\A	Matches only at the beginning of the string.
\Z	Matches only at the end of the string, or before a newline at the end.
\z	Matches only at the absolute end of the string.
\G	Matches at the position where the previous /g match ended. Useful for iterative matching.
\b	Matches a word boundary (position between \w and \W).
\B	Matches a non-word boundary.
\n, \t, \r, \f	Matches newline, tab, carriage return, form feed respectively.
\1, \2…	Backreferences: Match the same text captured by the Nth group.
\k	Named backreference: Matches text captured by a named group.

Example demonstrating anchors and word boundaries:

#!/usr/bin/perl
use strict;
use warnings;

my $string = "The cat scattered its catapult.";

# Matches 'cat' as a whole word
if ($string =~ /\bcat\b/) {
    print "Found whole word 'cat'. Matched: '$&'\n";
}

# Matches 'cat' NOT as a whole word (e.g., part of 'catapult')
if ($string =~ /\Bcat\B/) {
    # This would match 'cat' inside 'scattered' if it were 'scatered'.
    # For 'catapult', \bcat\B would match 'cat' at the start of 'catapult'.
    print "Found 'cat' within a word (e.g., 'scattered' if applicable). Matched: '$&' (if any)\n";
}

# Matches 'cat' at the beginning of 'catapult'
if ($string =~ /\bcat\B/) {
    print "Found 'cat' at start of a longer word. Matched: '$&'\n"; # Matches 'cat' in 'catapult'
}

Output will vary based on matches. For the provided string and regexes:

Found whole word 'cat'. Matched: 'cat'
Found 'cat' at start of a longer word. Matched: 'cat'

The \G Assertion for Continuous Matching

The \G assertion is used with the /g modifier to anchor a match to the position where the last match ended (pos($string)). This is useful for parsing or tokenizing strings piece by piece.

#!/usr/bin/perl
use strict;
use warnings;

my $text = "key1=value1;key2=value2;key3=value3";
while ($text =~ m/\G(?<key>\w+)=(?<value>\w+);?/gc) {
    print "Key: $ENV{'+'}{'key'}, Value: $ENV{'+'}{'value'}\n";
}
# The /c modifier prevents reset of pos() on final non-match of the loop condition.

Output:

Key: key1, Value: value1
Key: key2, Value: value2
Key: key3, Value: value3

Perl’s regular expressions are incredibly versatile. For complex patterns, the /x modifier is highly recommended for adding comments and whitespace to improve readability. Mastering regular expressions is a key skill for effective Perl programming. Consult perldoc perlre for the comprehensive guide.