Sed Loops

Sed - Loops and Branching

Like traditional programming languages, sed provides mechanisms for controlling the flow of execution, specifically through labels and branching commands. This allows for creating loops and conditional execution of sed commands, enabling more complex text processing logic.

Labels

A label in sed marks a specific point in your script. You can then use branching commands to jump to this label. A label is defined by a colon (:) followed by the label name. Label names can contain any characters except a newline.

Example label definitions:

:mylabel
:start_loop
:process_data
:end_section

Branching Commands

sed offers two primary branching commands:

• b [label] (Branch): Unconditionally jumps to the specified label. If no label is provided, sed jumps to the end of the script, effectively starting the next cycle of reading a new line (if any).
• t [label] (Test and Branch): Jumps to label only if a s/// (substitute) command has made a successful substitution since the last input line was read or the last t command was executed. If no label is provided, it jumps to the end of the script. This is useful for looping until a condition (no more substitutions) is met.

Example: Processing Paired Lines with a Loop and Branch

Let’s consider a books.txt file where each book’s title and author are on separate, consecutive lines. We want to combine them onto a single line, separated by a comma, and then prepend a hyphen (- ) only if the combined line contains “Paulo”.

Assume `books.txt` looks like this:

A Storm of Swords
George R. R. Martin
The Two Towers
J. R. R. Tolkien
The Alchemist
Paulo Coelho
The Fellowship of the Ring
J. R. R. Tolkien
The Pilgrimage
Paulo Coelho
A Game of Thrones
George R. R. Martin

The sed script to achieve this:

$ sed -n ' \
N; # Append next line to pattern space, creating 'line1\nline2'
 s/\n/, /; # Replace newline with ', '
 /Paulo/!b PrintLine; # If 'Paulo' NOT found, branch to PrintLine
 s/^/- /; # If 'Paulo' found (no branch), prepend '- '
:PrintLine
 p # Print the (possibly modified) pattern space
' books.txt

Output:

A Storm of Swords, George R. R. Martin
The Two Towers, J. R. R. Tolkien
- The Alchemist, Paulo Coelho
The Fellowship of the Ring, J. R. R. Tolkien
- The Pilgrimage, Paulo Coelho
A Game of Thrones, George R. R. Martin

Demystifying the Script:

• sed -n '...' books.txt: The -n option suppresses automatic printing. We’ll use p to print explicitly.
• N;: This is the Next command. It appends the next line of input into the pattern space, separated by a newline character (\n). So, if pattern space held “Title”, after N it holds “Title\nAuthor”. This command implicitly creates a loop to process pairs of lines.
• s/\n/, /;: The substitute command replaces the embedded newline (introduced by N) with a comma and a space. Pattern space becomes “Title, Author”.
• /Paulo/!b PrintLine;: This is a conditional branch. /Paulo/ is an address that matches lines containing “Paulo”. The ! negates this, so it means “if the pattern space does not contain ‘Paulo’”. If true, the b PrintLine command branches (jumps) to the label :PrintLine. If false (i.e., “Paulo” is in the pattern space), execution continues to the next command.
• s/^/- /;: This substitution command is only reached if the previous branch was not taken (meaning “Paulo” was found). It prepends a hyphen and a space (- ) to the beginning of the pattern space (^).
• :PrintLine: This defines the label PrintLine.
• p: The print command prints the current content of the pattern space.

This script demonstrates how N can be used to process multiple lines together and how b with a label can control the flow based on pattern matching. The original example used h;n;H;x which is another way to join lines using the hold space, more suitable for complex multi-line records. The N command is often simpler for joining adjacent lines.

Alternative using Hold Space (closer to original example): The original example used h;n;H;x to combine lines. This sequence works as follows:

1. h: Copy pattern space (current line, e.g., “Title”) to hold space.
2. n: Read next line (e.g., “Author”) into pattern space (overwriting “Title”).
3. H: Append pattern space (“Author”) to hold space. Hold space becomes “Title\nAuthor”.
4. x: Exchange pattern space and hold space. Pattern space now contains “Title\nAuthor”.

Then the script would proceed similarly: s/\n/, /; /Paulo/!b Print; s/^/- /; :Print; p.

The choice between N and h;n;H;x depends on the specific multi-line processing task. N is simpler for fixed pairs, while hold space operations offer more flexibility for complex scenarios.

Loops and branching, especially when combined with hold space operations, are among sed’s most advanced features. They require careful thought but provide immense power for sophisticated text transformations.