Linux Pipes and Redirection
Pipes, Redirection, Stdin, Stdout and Stderr
One of the most useful functions you will come across in your quest to learn Linux is its ability to redirect output to screen or file. You will learn to pass output from one command to another command for further processing.
Information may be displayed to the screen and sent to a file almost simultaneously.
Before we look at some of these command, we need to understand the terms "Standard Input", "Standard Output" and "Standard Error".
Standard Input, or "stdin" as it is more commonly known is a name given to a channel of input. Stdin is basically information that is entered via your keyboard. Standard input is assigned the channel number "0".
Standard Output, or "stdout" is the name given to output that is normally sent to your screen. Standard output is assigned the channel number "1".
Standard Error, or "stderr" is the output that is normally sent to your screen. As you have probably guessed, this output contains errors normally from programs. The channel number "2" has been assigned to this channel.
Redirection of Channels
Redirection of standard output is generally achieved by the use of the ">". operator. Lets take a look at a simple example:
$ ls -l > fileout.txt $ ls -l total 4 -rw-rw-r-- 1 john john 58 Feb 3 14:33 fileout.txt
In the above example, the output from the command "ls -l" is redirected to a file called "fileout.txt". If this file had already existed, then it would be overwritten. If the command failed to produce any output, then the file is still created and would remain an empty file.
If we simply wanted to append output to an already existing file, then we can use the operator ">>":
$ echo "I have been appended" >> fileout.txt $ cat fileout.txt -rw-rw-r-- 1 john john 0 Feb 3 14:33 fileout.txt I have been appended
As we can see from the above example, the output from the "echo" command was appended to the fileout.txt. As well as being able to redirect the output from a program we can redirect the standard input channel using the operator "<".
$ cat fileout.txt total 0 -rw-rw-r-- 1 john john 0 Feb 3 14:33 fileout.txt I have been appended $ wc -l < fileout.txt 3
Now lets combine both operators in one example:
$ wc -l < fileout.txt > filecount.txt $ ls -l total 8 -rw-rw-r-- 1 john john 2 Feb 3 15:12 filecount.txt -rw-rw-r-- 1 john john 80 Feb 3 14:49 fileout.txt $ cat filecount.txt 3
So far we have seen the "standard input" and "standard output" channels used with redirection. It is also possible to redirect our other channel "standard error". To redirect a different channel to another, we use the ">&" operator. So we could specify our command similar to "command > command.log 2>&1". Here we are redirecting the "stdout" and "stderr" to the command.log.
In Linux there is a way to pass the output from one command to the input of another command by means of a "pipe". To connect output from one command to another we use the "|" vertical bar operator.
$ ls -l > pipe.txt $ cat pipe.txt | wc -l 2
In the above example, we created the file "pipe.txt" from the output of the "ls" command. Next we use the "cat" command to output its content, however, we use the "|" to pass the output to the program "wc" for processing. Multiple pipes can be combined together along with redirection.
$ tail /etc/passwd | sort | grep "john" > test.txt $ cat test.txt john:x:1000:1000:john,,,:/home/john:/bin/bash
In this example, we used the tail command to read the last few lines of the "/etc/passwd" file. The output was then passwd to the "sort" command and then the output from that command was passed to the "grep"command. Finally the results were then redirected to the file "test.txt".
Another very useful command is the tee command. This command will allow you to send output from one command to your screen and to a file.
$ ls -al | tee tee.txt total 8 drwxrwxr-x 2 john john 4096 Feb 3 16:21 . drwxr-xr-x 63 john john 4096 Feb 3 14:33 .. $ ls -l total 4 -rw-rw-r-- 1 john john 97 Feb 3 16:21 tee.txt
In this example, we issued the "ls -al" command and piped the output to the "tee" command along with the filename tee.txt. As you can see from the output, the tee command sent the output to screen and also to the specified file.