Pipe Dreams





Last Updated on 05/27/2017 by dboth

Pipes are extremely important and even critical to our ability to do very amazing things on the command line, that I think it is important to recognize that they were invented by Douglas McIlroy during the early days of Unix. Thanks, Doug!

Notice the use of pipes in this simple command line program that lists each logged-in user a single time no matter how many logins they have active .

who | awk '{print $1}' | sort | uniq

Pipes are the glue that holds these command line programs together. Pipes allow the Standard Output from one command to be “piped” to the Standard Input of the next command.

Think about how this program would have to work if we could not pipe the data stream from one command to the next. The first command would perform its task on the data and then the output from that command would have to be saved in a file. The next command would have to read the stream of data from the intermediate file and perform its modification of the data stream, sending its own output to a new, temporary data file. The third command would have to take its data from the second temporary data file and perform its own manipulation of the data stream and then store the resulting data stream in yet another temporary file. At each step the data file names would have to be transferred from one command to the next in some way.

I cannot even stand to think about that it is so complex.

More Philosophy

It is only with the use of pipes that several tenets of the Linux Philosophy make sense.

Each of these programs is small [is beautiful] and each does one thing well. They are also filters, that is they take Standard Input, process it in some way and then send the output to Standard Output.

The use of Standard I/O for input and output is one of the secondary tenets of the Linux Philosophy. It is only through the use of pipes that implementation of these programs as filters to send processed data streams from their own Standard Output to the Standard Input of the program above is even possible.