Pipes

Linux Pipes and Redirection

Pipes, Redirection, Stdin, Stdout and Stderr


One of the most useful functions you will come across in your quest to learn Linux is its ability to redirect output to screen or file. You will learn to pass output from one command to another command for further processing. Information may be displayed to the screen and sent to a file almost simultaneously.

Before we look at some of these command, we need to understand the terms "Standard Input","Standard Output"and "Standard Error".



Standard Input


Standard Input, or "stdin" as it is more commonly known is a name given to a channel of input. Stdin is basically information that is entered via your keyboard. Standard input is assigned the channel number "0".


Standard Output


Standard Output, or "stdout" is the name given to output that is normally sent to your screen. Standard output is assigned the channel number "1".


Standard Error


Standard Error, or "stderr" is the output that is normally sent to your screen. As you have probably guessed, this output contains errors normally from programs. The channel number "2" has been assigned to this channel.


Channel Number Description Name Device
0 Standard Input stdin Keyboard
1 Standard Output stdout Screen
2 Standard Error stderr Screen

Redirection of Channels


Redirection of standard output is generally achieved by the use of the ">". operator. Lets take a look at a simple example:


john@john-desktop:~/test_examples$ ls -l > fileout.txt
john@john-desktop:~/test_examples$ ls -l
total 4
-rw-rw-r-- 1 john john 58 Feb 3 14:33 fileout.txt
john@john-desktop:~/test_examples$


In the above example, the output from the command "ls -l" is redirected to a file called "fileout.txt". If this file had already existed, then it would be overwritten. If the command failed to produce any output, then the file is still created and would remain an empty file.

If we simply wanted to append output to an already existing file, then we can use the operator">>":


john@john-desktop:~/test_examples$ echo "I have been appended" >> fileout.txt
john@john-desktop:~/test_examples$ cat fileout.txt
-rw-rw-r-- 1 john john 0 Feb 3 14:33 fileout.txt
I have been appended


As we can see from the above example, the output from the "echo" command was appended to the fileout.txt. As well as being able to redirect the output from a program we can redirect the standard input channel using the operator "<".


john@john-desktop:~/test_examples$ cat fileout.txt
total 0
-rw-rw-r-- 1 john john 0 Feb 3 14:33 fileout.txt
I have been appended
john@john-desktop:~/test_examples$ wc -l < fileout.txt
3


Now lets combine both operators in one example:


john@john-desktop:~/test_examples$ wc -l < fileout.txt > filecount.txt
john@john-desktop:~/test_examples$ ls -l
total 8
-rw-rw-r-- 1 john john 2 Feb 3 15:12 filecount.txt
-rw-rw-r-- 1 john john 80 Feb 3 14:49 fileout.txt
john@john-desktop:~/test_examples$ cat filecount.txt
3


So far we have seen the "standard input" and "standard output" channels used with redirection. It is also possible to redirect our other channel "standard error". To redirect a different channel to another, we use the ">&" operator. So we could specify our command similar to "command > command.log 2>&1". Here we are redirecting the "stdout" and "stderr" to the command.log.



Pipes


In Linux there is a way to pass the output from one command to the input of another command by means of a "pipe". To connect output from one command to another we use the "|" vertical bar operator.


john@john-desktop:~/test_examples$ ls -l > pipe.txt
john@john-desktop:~/test_examples$ cat pipe.txt | wc -l
2


In the above example, we created the file "pipe.txt" from the output of the "ls" command. Next we use the "cat" command to output its content, however, we use the "|" to pass the output to the program "wc" for processing. Multiple pipes can be combined together along with redirection.


john@john-desktop:~/test_examples$ tail /etc/passwd | sort | grep "john" > test.txt
john@john-desktop:~/test_examples$ cat test.txt
john:x:1000:1000:john,,,:/home/john:/bin/bash


In this example, we used the tail command to read the last few lines of the "/etc/passwd" file. The output was then passwd to the "sort" command and then the output from that command was passed to the "grep"command. Finally the results were then redirected to the file "test.txt".


tee command


Another very useful command is the tee command. This command will allow you to send output from one command to your screen and to a file.


john@john-desktop:~/test_examples$ ls -al | tee tee.txt
total 8
drwxrwxr-x 2 john john 4096 Feb 3 16:21 .
drwxr-xr-x 63 john john 4096 Feb 3 14:33 ..
john@john-desktop:~/test_examples$ ls -l
total 4
-rw-rw-r-- 1 john john 97 Feb 3 16:21 tee.txt


In this example, we issued the "ls -al" command and piped the output to the "tee" command along with the filename tee.txt. As you can see from the output, the tee command sent the output to screen and also to the specified file.