IntermediateText processing

Sed Basics

Use sed's s command, address ranges, d and p commands, in-place editing with -i, and basic multiline patterns to transform text streams.

BashIntermediate11 min read

Recommended first

Grep Patterns

By the end of this lesson you will be able to:

Write s/pattern/replacement/flags substitution commands
Address specific lines or ranges for targeted edits
Delete and print lines selectively with d and p
Edit files in place safely with -i (and a backup)
Explain when multiline patterns are needed

sed — the stream editor — reads a file (or stdin) line by line, applies a script of editing commands, and writes the result to stdout. It is the right tool when you need to transform text in a predictable, scriptable way: replacing a configuration value, stripping comments, normalising whitespace, or renaming a symbol across many files at once.

The s command: substitution

The core of sed is the substitution command:

sed 's/old/new/' file.txt          # replace first occurrence per line
sed 's/old/new/g' file.txt         # replace ALL occurrences per line (global flag)
sed 's/old/new/2' file.txt         # replace only the 2nd occurrence per line
sed 's/old/new/gi' file.txt        # global + case-insensitive (GNU sed)

The delimiter is traditionally / but can be any character — useful when the pattern contains slashes:

sed 's|/usr/local|/opt|g' paths.txt
sed 's#http://old.example.com#https://new.example.com#g' links.html

Capture groups use \(…\) in BRE (default) or (…) with sed -E, and back-references use \1, \2:

# Swap first and last name (BRE)
echo "Smith, Alice" | sed 's/\([^,]*\), \(.*\)/\2 \1/'
# Alice Smith

# With ERE (-E flag)
echo "Smith, Alice" | sed -E 's/([^,]+), (.+)/\2 \1/'

Addressing lines

By default a sed command applies to every line. An address restricts it:

sed '3s/old/new/' file.txt         # only line 3
sed '2,5s/old/new/' file.txt       # lines 2 through 5
sed '/pattern/s/old/new/' file.txt # only lines matching pattern
sed '/start/,/end/s/old/new/' file.txt  # lines between start and end patterns
sed '$s/old/new/' file.txt         # only the last line

Address ranges using two patterns are especially powerful for config file editing:

# Replace "debug=false" only between [logging] and the next blank line
sed '/\[logging\]/,/^$/s/debug=false/debug=true/' config.ini

Deleting and printing lines

d deletes matching lines from the output; p prints them (useful with -n to suppress default output):

sed '/^#/d' script.sh              # strip comment lines
sed '/^[[:space:]]*$/d' file.txt   # strip blank lines

# Print only matching lines (like grep)
sed -n '/ERROR/p' app.log

# Print a range of lines (lines 10 to 20)
sed -n '10,20p' file.txt

-n suppresses default output. Normally sed prints every line (after any edits). With -n, only lines with an explicit p command are printed. This lets you use sed as a selective filter — but unlike grep, you can also transform the lines you select.

In-place editing: -i

-i edits the file in place instead of writing to stdout. On GNU sed you can optionally provide an extension to create a backup:

sed -i 's/version=1.0/version=2.0/' config.cfg       # in-place, no backup
sed -i.bak 's/foo/bar/g' file.txt                     # creates file.txt.bak first

macOS/BSD sed requires a suffix with -i, even if empty: sed -i '' 's/foo/bar/' file. The GNU form sed -i (no suffix) fails on macOS. For cross-platform scripts, always provide the suffix — use sed -i.bak or use sed -i '' and test on both platforms.

In-place editing across many files is a common use case:

# Replace an old domain name in all config files
find . -name "*.conf" | xargs sed -i.bak 's/old.example.com/new.example.com/g'

Multiline patterns

By default, sed operates one line at a time and . does not match newlines. For multiline work, use the N command to pull the next line into the pattern space:

# Join lines that end with a backslash to the next line
sed ':loop; /\\$/ { N; s/\\\n//; b loop }' file.txt

This is an advanced technique — for most multiline tasks, awk (next lesson) or Python is a cleaner choice.

Check your understanding

Do it yourself

# Create a test file
cat > /tmp/sed-test.txt <<'EOF'
# This is a comment
name=Alice
version=1.0
# Another comment
name=Bob
version=1.0
EOF

# Strip comments
sed '/^#/d' /tmp/sed-test.txt

# Replace version
sed 's/version=1.0/version=2.0/g' /tmp/sed-test.txt

# Replace with backup (adjust for your OS)
cp /tmp/sed-test.txt /tmp/sed-test.bak
sed -i.bak 's/Alice/Charlie/' /tmp/sed-test.txt
cat /tmp/sed-test.txt

Where to go next

You can now transform text streams with substitutions and address ranges. The next lesson — awk basics — takes text processing further: structured field splitting, accumulation, and BEGIN/END blocks for full-featured text programs.

Finished reading? Mark it complete to track your progress.

Grep Patterns

Use grep with BRE and ERE, anchors, character classes, and flags like -i, -v, -l, -c, -r, and --include to search text precisely.

Awk Basics

Use awk to split fields, apply per-record logic with BEGIN/END blocks, use NR/NF, accumulate sums, and build small text programs.