unix sed command utility
DESCRIPTION
About sed commandTRANSCRIPT
Unix Sed Command Utility
Internal Use 2
Contents
Contents .........................................................................................................................2 Sed - An Introduction.....................................................................................................3 Sed Utility ......................................................................................................................4
syntax for basic sed commands: ................................................................................4 Forward slash .............................................................................................................4 s -- Search and replace ...............................................................................................4 Backslash ...................................................................................................................4 g flag ..........................................................................................................................5 Ampersand .................................................................................................................5
Applications of sed command........................................................................................5 Deleting lines in a file using sed command ...............................................................6 Deleting the last line in the input file.........................................................................6 To delete the lines 2 through 4 in the above file, using sed command ......................6 Replace or substitute all occurrences of a pattern using sed command.....................6 Delete all occurrences of a pattern in a file using sed command...............................7 To remove all occurrences of the character space using sed command ....................7 Deleting lines which matches a pattern using sed command.....................................8 Deleting the first line which matches a pattern using sed command.........................8 Deleting the last line which matches a pattern using sed command..........................8
Unix Sed Command Utility
Internal Use 3
Sed - An Introduction
The most powerful text filtering tools in the UNIX environment are sed and awk. They let shell programmers easily edit text files and filter the output of other commands using regular expressions.
sed (which stands for stream editor) was created as an editor exclusively for executing scripts. As its name implies, sed is stream oriented, thus all the input you feed into it passes through and goes to STDOUT and it does not change the input file.
Unix Sed Command Utility
Internal Use 4
Sed Utility sed is short for stream editor.The traditional, modern-day definition of a text editor is an interactive application that can be used to create and edit text files. sed is also a text editor, but it is a command-line utility instead of an interactive utility, which makes it an extremely powerful tool for batch editing. sed is commonly used in UNIX shell scripts to filter large sets of text files.
syntax for basic sed commands:
sed s/REGULAREXPRESSION/REPLACEMENTSTRING/flags INPUT_FILE
Forward slash All of the special characters explained earlier for use with grep also work in sed. To use sed, however, you must learn some additional syntax. A basic expression in sed is composed of four parts, separated by forward slashes (/).
s -- Search and replace The s indicates you want to execute a search and replace. The forward slashes are used to bind regular expressions in sed. For instance, if you simply want to replace the term logfile with logfile.txt, you would run the following command:
sed s/logfile/logfile.txt/ sed.txt For future reference, the output can be captured or sent into a new file. For example, to send the output to sed_new.txt, run this command:
sed s/logfile/logfile.txt/ sed.txt > sed_new.txt
Backslash The backslash (\) is called the escape character, because it escapes the next character from the regular expression interpretation. More simply, putting a backslash before a special character makes the character a plain item instead of a command item. This is important because many files, especially when you are writing code, make extensive use of the same characters that are used to execute a regular expression. In your sed.txt file, you'll notice
Unix Sed Command Utility
Internal Use 5
that the dollar sign is used. If you want to replace $project but not project, you need to use the escape character in your search and replace:
sed s/\$project/\$project_name/ sed.txt
g flag Execute the same sed command, but this time tack a g on the end.Introducing filters and regular expressions sed s/project/project_name/g sed.txt This time, both instances of project were changed to project_name on the first
Ampersand The ampersand (&) represents the string that was matched by your regular expression. In other words, if [a-z]*: turned out to be project: on a particular line, the ampersand would hold that value. sed s/[a-z]*:/new_\&/g sed.txt
Applications of sed command Sed command can be used to search and manipulate input text in many ways, though mainly used for substitution, it can also do numerous other pattern matching tasks like searching and printing lines which matches a pattern, removing lines which matches a pattern, deleting specific lines in the input file. The text file used to illustrate the below sed commands is shown below. bash-3.00# cat textfile.txt This is line 1 This is line 2 This is line 3 This is line 4 This is line 5
Unix Sed Command Utility
Internal Use 6
Deleting lines in a file using sed command Deleting the first line in the input file bash-3.00# sed '1d' textfile.txt This is line 2 This is line 3 This is line 4 This is line 5
Deleting the last line in the input file bash-3.00# sed '$d' textfile.txt
This is line 1 This is line 2 This is line 3 This is line 4
To delete the lines 2 through 4 in the above file, using sed command bash-3.00# sed '2,4d' textfile.txt
This is line 1 This is line 5
the 'd' flag is used to delete lines.
Replace or substitute all occurrences of a pattern using sed command
In the above text file, if you need to replace all occurrences of the pattern "line" with the text "line number", the following sed command can be used (the 's' flag in the sed command is used to substitute text which matches a pattern).
bash-3.00# sed 's/line/line number/g' textfile.txt
This is line number 1 This is line number 2 This is line number 3 This is line number 4
Unix Sed Command Utility
Internal Use 7
This is line number 5
Note: The /g flag in the above command is used to instruct sed to substitute globally i.e. all occurrences of the input pattern in the line will be replaced and as a result all words matching the pattern in the input file will be substituted by the other, without /g only the first word matching the pattern in a line will be replaced.
Delete all occurrences of a pattern in a file using sed command
This is a simple application of the /s flag where the target string is none.
For example, to remove all occurrences of word "line" in the above file, the sed command would be
bash-3.00# sed 's/line//' textfile.txt This is 1 This is 2 This is 3 This is 4 This is 5
Note that the word to be replaced upon matching the pattern is none in the above command ('/s/line//')
To remove all occurrences of the character space using sed command
bash-3.00# sed 's/ //g' textfile.txt
Thisisline1 Thisisline2 Thisisline3 Thisisline4 Thisisline5
Also note that if you don't use single quotes in the above sed command, then the space pattern should be enclosed in quotes for the command to work.
bash-3.00# sed s/' '//g textfile.txt
Thisisline1 Thisisline2 Thisisline3 Thisisline4 Thisisline5
Unix Sed Command Utility
Internal Use 8
Deleting lines which matches a pattern using sed command
To delete lines which matches a pattern, use the /d flag along with the pattern to be matched, for example, to delete lines matching the pattern "line 1" in the above file, the command would be
bash-3.00# sed '/line 1/d' textfile.txt
This is line 2 This is line 3 This is line 4 This is line 5
Now using the above commands, one can also delete the first and last line in a file which matches a pattern.
Deleting the first line which matches a pattern using sed command
bash-3.00# sed '/line/{1d;}' textfile.txt This is line 2 This is line 3 This is line 4 This is line 5
In the above command, '1d' is used in braces to indicate that its a command (to delete the specific line, which matches a pattern), this is the way to separate sed flags in command/scripts.
Deleting the last line which matches a pattern using sed command
bash-3.00# sed '/line/{$d;}' textfile.txt
This is line 1 This is line 2 This is line 3 This is line 4