Sed Special Characters

Understanding Special Characters in sed

The Stream Editor, sed, utilizes special characters that act as commands or have unique meanings within scripts. Mastering these can significantly enhance your text processing capabilities. This section focuses on two key special characters: = and &.

The = Command: Displaying Line Numbers

The = command is used to print the line number of the current line to standard output. It’s often followed by a newline character, and then sed proceeds to print the line itself (unless default output is suppressed).

Syntax:

[/pattern/]=
[address1[,address2]]=

Let’s assume we have a file named books.txt with the following content:

1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

Example 1: Print line numbers for all lines.

$ sed '=' books.txt

Output:

1
1) A Storm of Swords, George R. R. Martin, 1216
2
2) The Two Towers, J. R. R. Tolkien, 352
3
3) The Alchemist, Paulo Coelho, 197
4
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5
5) The Pilgrimage, Paulo Coelho, 288
6
6) A Game of Thrones, George R. R. Martin, 864

Explanation: For each line, sed first prints the line number, followed by a newline, and then prints the line content itself due to its default behavior.

Example 2: Print line numbers for a specific range of lines (e.g., first four lines).

$ sed '1,4=' books.txt

Output:

1
1) A Storm of Swords, George R. R. Martin, 1216
2
2) The Two Towers, J. R. R. Tolkien, 352
3
3) The Alchemist, Paulo Coelho, 197
4
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

Explanation: Line numbers are printed for lines 1 through 4. Lines 5 and 6 are printed without preceding line numbers because they are outside the specified address range for the = command.

Example 3: Print line numbers only for lines matching a pattern.

$ sed '/Paulo/=' books.txt

Output:

1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
3
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

Explanation: When a line contains the pattern “Paulo”, its line number is printed before the line itself.

Example 4: Counting the total number of lines (simulating wc -l).

$ sed -n '$=' books.txt

Output:

6

Explanation: The $ address refers to the last line. The = command prints its line number. The -n option suppresses sed’s default behavior of printing each line. Thus, only the line number of the last line (which is the total count of lines) is printed.

The & Character: Reusing the Matched Pattern

The ampersand & is a special character used primarily in the replacement part of the substitution command (s/pattern/replacement/). It represents the entire string that was matched by the pattern.

Example 1: Prepending text to a matched pattern.

Suppose we want to prepend “Book ID: ” to the numerical ID at the beginning of each line in books.txt.

$ sed 's/^[0-9]/Book ID: &/' books.txt

Here, ^[0-9] matches the first digit at the beginning of the line. [[:digit:]] is a more portable POSIX character class for a digit, so s/^[[:digit:]]/Book ID: &/' would also work.

Output:

Book ID: 1) A Storm of Swords, George R. R. Martin, 1216
Book ID: 2) The Two Towers, J. R. R. Tolkien, 352
Book ID: 3) The Alchemist, Paulo Coelho, 197
Book ID: 4) The Fellowship of the Ring, J. R. R. Tolkien, 432
Book ID: 5) The Pilgrimage, Paulo Coelho, 288
Book ID: 6) A Game of Thrones, George R. R. Martin, 864

Explanation: The pattern ^[0-9] (or ^[[:digit:]]) matches the leading digit (e.g., ‘1’, ‘2’). The & in the replacement string refers to this matched digit. So, ‘1’ becomes ‘Book ID: 1’.

Example 2: Adding context around the page numbers.

Let’s say the last number in each line of books.txt represents page count. We want to format it as “Pages: [number]”.

$ sed 's/[0-9]*$/Pages: &/' books.txt

Alternatively, using POSIX character class: sed 's/[[:digit:]]*$/Pages: &/' books.txt

Output:

1) A Storm of Swords, George R. R. Martin, Pages: 1216
2) The Two Towers, J. R. R. Tolkien, Pages: 352
3) The Alchemist, Paulo Coelho, Pages: 197
4) The Fellowship of the Ring, J. R. R. Tolkien, Pages: 432
5) The Pilgrimage, Paulo Coelho, Pages: 288
6) A Game of Thrones, George R. R. Martin, Pages: 864

Explanation: The pattern [0-9]*$ (or [[:digit:]]*$) matches a sequence of zero or more digits at the end of the line (the page number). The & then inserts this matched page number into the replacement string.

Practical Application:

The & character is invaluable for tasks like wrapping matched text with HTML tags, adding quotes around specific words, or reformatting parts of a line while preserving the original matched content. For instance, converting all occurrences of ‘Warning:’ to ‘Warning:’.

Further Resources:

• GNU sed manual: Detailed information on sed commands and special characters. (Search for ‘GNU sed manual’)
• Regular-Expressions.info: A comprehensive resource for understanding regular expressions, which are crucial for using sed effectively.