Linux and the Unix Philosophy
6.3 The Linux environment: Using programs as filters
You may be wondering what it means for a program to act as a filter. Linux programmers follow a set of unwritten rules that simplify designing software that behaves in this manner. To help clarify these rules, I've included several guidelines here. Before we discuss these, however, it is necessary to provide a short explanation of a concept in Linux called stdio.
When a program is invoked under Linux, it normally has three standard I/O channels open to it known as stdin, stdout, and stderr, hence the name stdio. What is connected to the other ends of these I/O channels depends on how the program was invoked. In the default case, stdin collects user input when the program is invoked, and stdout sends any program output to the user's display screen. Any output sent to stderr also appears on the display screen, but this data is normally considered "out of band" or error information.
An interesting feature of Linux stdio is that the devices connected to the I/O channels are not "hard-wired" to the program. At the time of invocation, the user may specify that data will originate from or be sent to places other than the user's terminal. For example, stdin may come from a file, another program via a Linux pipe, or even a satellite link to a system on the other side of the world. Similarly, if the user wanted to save stdout in a file for later perusal, he or she could direct the Linux shell to place the output there. This provides for enormous flexibility with respect to the source and destination of the data.
How Linux programmers deal with stdio has a significant impact on the ability of a program to function as a filter. If the program is written correctly, then all the adaptability of stdio can be used. Otherwise, the user is likely to be locked within a CUI. The key, then, is to write the software so that it employs stdio. Here are three important guidelines:
1. Use stdin for data input
Programs that obtain their input from stdin assume that their data could come from anywhere. Indeed it could. By avoiding "hard-wiring" the input channel, you make it easy for the user to specify where the input will come from when invoking the program. Its source could be the keyboard, a file, another Linux program, or even a CUI.
2. Use stdout for data output
As stdin usage allows your program to accept input data from anywhere, the use of stdout allows your program's output to be sent anywhere. "Anywhere" here may mean the user's screen, a file, a printer, or even a digital speech synthesizer. The choice is up to the user, and it can be whatever is appropriate when the program is run.
3. Use stderr for out-of-band information
Error messages and other warnings should be sent to the user via stderr. They should never be part of the data stream sent to stdout. One reason for this is that the user may choose to capture error messages in a separate file or perhaps view them on the terminal immediately. Sending error messages on the same I/O channel as stdout can cause confusion further down the line. Remember that Linux commands are seldom used alone.
Notice how this approach differs from that of other operating systems. Applications running on most other systems tend to hard-wire everything. They assume that there will always be a user sitting at the keyboard. They may ask the user if he or she would like to send the output to a file, but they seldom offer the individual this choice unless a conscious effort was made to include this capability.
Hard-wiring the I/O implies that you know all possible users for your program. This is sheer egotism. In an earlier chapter we stressed that everyone is on a learning curve. No one can predict how his or her software will always be used. The best you can do is make the interface to your programs flexible enough to deal with as many eventualities as exist today. Beyond that, let tomorrow take care of tomorrow.
After having spent many years in a software engineering environment, I once took a position in a telephone support center where I answered customer questions about software-in many cases my software. It was most enlightening to speak with customers who depended on the software to do their jobs. I listened intently to hundreds of them tell stories of how they were doing the unbelievable with my software. My most common reaction was, "It wasn't meant to do that!"
Recent network security break-ins have made it painfully clear that malicious users often attempt to do things that the software was never intended to do. These break-ins usually result from exploitation of buffer overflows and other programming errors that leave systems vulnerable. While the programmer may have intended for a program operate in a certain way, a system cracker will often invent strange and not-so-wonderful ways to use it to gain unauthorized access.
You never know what people are going to do with your software. Never, ever assume that they will use it solely for the purpose you intended. You may think that you're writing a simple sort program, a word processor, or a file compression routine. You'll soon discover that someone is using the sort program to translate ASCII to EBCDIC, the word processor has become a public access bulletin board, and the file compression routine is being used to digitize Gone with the Wind for downloading over the local cable TV system.
It's easier to avoid developing programs with CUIs if you keep in mind that all programs are filters. When you assume that the receptacle of a program's data flow might be another program instead of a human being, you eliminate those biases we all have in trying to make an application user friendly. You stop thinking in terms of menu choices and start looking at the possible places your data may eventually wind up. Try not to focus inward on what your program can do. Look instead at where your program may go. You'll then begin to see the much larger picture of which your program is a part.
When regarding programs as filters, software designers break their applications down into collections of smaller programs, each of which performs a function of the application. Not only do these small programs communicate well with each other, they lack most of the bloat required to make their user interfaces "bulletproof." In a sense, their "users" are other programs. These programs will ultimately yield higher performance. They are not constrained by human capabilities that will fall behind when faster architectures are introduced in the future. And programs, as we said before, don't complain, develop attitudes, or call in sick.
Now, if we could just get the programmers not to complain, develop attitudes, or call in sick!