The Unix and Linux Philosophy





Last Updated on 08/28/2023 by dboth

Unix has a philosophy that underlies the entire operating system and which strongly affects the way in which administrators — and advanced users — interact with it. This philosophy is the parent of the Linux Philosophy. It is called “The Unix Way.”

Whether we know it or not, most of us have some sort of philosophy of life. That may be as simple as, “Be kind to others,” or it might be a very complex life philosophy.

Many companies have some sort of philosophy as well. They may be unwritten or well documented. When I worked at IBM, from 1974 through 1995, the IBM philosophy was well documented and strongly ingrained in the culture. IBM’s philosophy covered its business practices and how employees, customers and suppliers were to be treated. The IBM prime directive, as it were, was to treat everyone fairly, with respect and dignity.

Before I started working with Linux in some depth, I had never known that it had a philosophy. I mean, what could a philosophy actually do for an operating system? After a bit of research, I discovered that all operating systems have a philosophy. I also learned that the philosophy of an operating system matters.

For example, the philosophy of Windows is very similar to that of Digital Equipment Corporation’s (DEC) VMS operating system. This amounted to “shield the users from everything that might get them into trouble.” Of course the reason for this similarity is that the prime developer of Windows NT was the same person who developed VMS. The philosophy of both operating systems was based on the underlying belief that users are afraid of computers and needed to be shielded from their complexity.

The MAC OS is not much different, although the Graphical User Interface (GUI) desktop takes an object-oriented approach that, in the opinions of many, provides a more consistent, integrated, and much smoother user experience.

Both MAC and Windows do provide a Command Line Interface (CLI) that allows low-level interaction between the user and the operating system. However the Windows CLI is very limited with few commands, at least when compared to Linux and Unix.

The MAC CLI is a BASH shell that provides a great deal of power just as it does on Linux. However it is very difficult to find. The shell is buried in an avalanche of menu choices that can only be found if you know it is there and are actively looking for it.

Radical Concept

When Unix was being developed in the late 1960’s and early 1970’s, the developers were intent upon building an operating system that was significantly different from the operating systems that preceded. The philosophy of Unix was markedly different from that of other operating systems. And the Linux Philosophy is quite naturally derived directly from the Unix philosophy.

I find the following quote from the section “Introduction to the Unix Philosophy” of the book “Linux and the Unix Philosophy”, by Mike Gancarz,quite informative.

“An operating system, by its nature, embodies the philosophy of its creators…”

The creators of the Unix operating system started with a radical concept: They assumed that the user of their operating system would be computer literate from the start. The entire Unix philosophy revolves around the idea that the users know what they are doing.”

Wow! Did they really say that! It sounds pretty elitist and exclusive.

Not really. Quite the opposite is true in fact.

Another favorite quote about Unix that also applies to Linux is often attributed to a person named Doug Gwyn, “Unix was not designed to stop its users from doing stupid things, as that would also stop them from doing clever things.” And therein lies the truth about Linux. It never assumes that the user is incompetent. It always assumes you know what you are doing and allows you do do anything you tell it to. Anything – whether it is unintentionally bad or not. That is a huge amount of power in the hands of a user.

Linux treats everyone equally and allows everyone the maximum amount of power. That is egalitarian. Other operating systems are elitist and exclusive because they withhold or hide their power behind an inflexible Graphical User Interface that allows one to do only what the developers think we should be allowed to do.

Enlightenment

Over the years a number of people have attempted to enlighten the rest of us when they codified various aspects of the Linux philosophy. Mike Gancarz first wrote “The Unix Philosophy” and then followed it up with “Linux and the Unix Philosophy.” These books list 9 major tenets and 10 lesser tenets.

Eric Raymond has 17 Unix rules in his book, “The Art of Unix programming.”

And Oregon State University has it’s own Linux Philosophy which I think nicely depicts an engineer’s view of Linux.

In addition to listing all of these tenets and rules here, along with all of their implications, I have listed links or books in the Bibliography from which more detailed information can be obtained. I leave it to you, dear reader, to look them up and find out what they mean. Believe me, it will be a very enlightening exercise.

The Terminal Case

The Linux philosophy is epitomized by the ease with which one can open a terminal emulator to access the CLI and its concomitant power. First there are the multiple virtual terminals that can be accessed using the Ctrl-Alt-F[1-7] keys.

Even the Linux GUI desktops whisper “Use the force” to all who use them. Linux has several fine GUI desktop environments from which to choose so that every user can choose the one she likes best. And yet the power of the CLI is only a click away with many choices for both the terminal emulator and the shell.

My favorite terminal emulator for the GUI desktop is Konsole. Konsole allows multiple terminal sessions in a GUI window using tabs to switch between terminal sessions. Of course you can use your favorite shell in all sessions or a different shell in each session, depending upon your needs.

For even more flexibility there is the screen program which provides for multiple shell sessions within a single terminal session. This is particularly useful when logging in remotely because if your connection fails the screen session continues along with all of the programs running in the various shells launched within the screen session.

The Linux CLI softly seduces; it tantalizes with its power. It’s flexibility drew me in. It changed me – certainly it changed the way I use the computer. I still use the GUI desktop but as much to provide me with the flexibility of multiple Konsole sessions with screen running in them as to run programs like LibreOffice, Firefox and Thunderbird.

Complete Control

Linux does not handhold. It assumes you know what you are doing when you type a command and it proceeds to execute that command without asking if you really want to. It gives you complete control.

Imagine for a moment the chaos and frustration that would result from attempting to use a nail gun that asked you if you really wanted to shoot that nail and would not allow you to pull the trigger until you said the word “yes” aloud. Linux allows you to use the nail gun as you choose. Other operating systems let you know that you can use nails but don’t tell you what tool is used to insert the nails let alone allow you to put your own finger on the trigger.

Yes, there is danger where there is great power. Used wisely that power can also be harnessed to accomplish many great things.

The Linux Prime Directive

I have come to believe that the summation of The Linux philosophy is quite simple.

  • Treat all users with respect.

This amounts to allowing each user to do things her own way with a wide choice of powerful tools. It means making flexibility, simplicity, and freedom the foremost considerations when designing and building software systems. It has resulted in the creation of software that is such a work of art that it is still beautiful and going strong after almost 45 years for Unix and for more than 20 years for Linux.

How does all this translate into the real world? I will tell you more of my views in future articles in which I discuss some of the individual tenets and their application to daily sysadmin tasks and other aspects of life with Linux.

The basic tenets of the Linux way are listed below along with a short explanation. In various places in the text of the DataBook I will refer to one or more of these.

The GNU utilities is an Open Source collection of the basic administrative programs required by any Unix or Linux operating system. These include programs to copy, move and delete files, manage disks, write text documents and programs and many other day to day operational activities that must be performed by a user or administrator.


The Nine Major Tenets

There are nine major tenets to the Unix and Linux philosophy.

  • Small is Beautiful
  • Each Program Does One Thing Well
  • Prototype as Soon as Possible
  • Choose Portability Over Efficiency
  • Store Data in Flat Text Files
  • Use Software Leverage
  • Use Shell Scripts to Increase Leverage and Portability
  • Avoid Captive User Interfaces
  • Make Every Program a Filter

There are also ten lesser tenets and some corollaries to the Linux Philosophy that are also important. Here is a quick command line program as an example that encompasses most of these nine major tenets.

who | awk ‘{print $1}’ | sort | uniq

This little command line program performs the very simple task of listing all users who are currently logged in while only listing each logged in user only once. It also eliminates extraneous data provided by the who command. I won’t cover all of the ways in which this CLI program conforms to the Linux Philosophy, but I could make a good case that it meets most of them.

The following are the main tenets of the Linux Philosophy.

Small is Beautiful

Each program should do one thing well.  This makes for efficient usage of system resources and it makes the programs easy to understand. Link many small programs together in ways that perform tasks unforeseen by the original programmers if each of the small programs.

CommandSource lines
who755
awk3412
sort2614
uniq302
TOTAL7083

Table 1: Lines of C code in each command in the CLI program above.

Each of the commands in the sample program is fairly small, and each performs a specific task. The sort command, for example does only one thing. It sorts the data stream sent to it via Standard Input and sends the results to Standard Output. It can perform numeric, alphabetic and alphanumeric sorts in forward and reverse order. But it does nothing else. It only sorts but it is very, very good at that. Because it is very small, having only 2614 lines of code, it is also very fast.

Large monolithic programs get feature bloat and lack the flexibility of many small, well written programs.

Write programs that do one thing and do it well

The programs that make up the GNU utilities, as well as many other Linux utility programs are almost invariably written to perform a single task; and each performs its assigned task very well. Large programs that perform many tasks cannot be written to cover every possibility that a user or administrator might want or need to perform. Using the capabilities of many small, functionally targeted programs and stringing them together with pipes and redirection allows for an amazing array of possibilities, far more creative and diverse than the original programmers could conceive.

Prototype as Soon as Possible

Change is inevitable, so when creating new code use high level languages, i.e., scripting languages like BASH and Perl. Get regular feedback from the customer to verify correct functioning of the program.

It may never need to use compiled language to write code when following these guidelines. Thus your code is Open and Discoverable to others.

“Open” means that the code can be read by a human being. “Discoverable” means that the logic and the functioning of the code is easily apparent.

Choose Portability Over Efficiency

Extreme efficiency requires OS and hardware explicit hacks and is rarely portable because of that. If the code  runs fast enough it meets your needs. In any event Next Month’s hardware will run faster.

The portability of shell scripts can be far more efficient in the long run than the perceived efficiency of writing a program in a compiled language—not even considering the time required to compile and test such a program—because they can run on many otherwise incompatible systems.

For example, I worked on the email system at the State of North Carolina for several years. I was responsible for a collection of Perl and BASH CGI programs that ran on a Red Hat Linux host. These programs provided remote users in all of our 100 counties and hundreds of large and small agencies to perform account maintenance for their users. I was asked to test the possibility of moving all of these programs to Red Hat running on an IBM Z-series mainframe.

I created a tarball from the existing programs, data, and configuration files, copied the tarball to a Red Hat instance on the mainframe and untarred it. I made one configuration change to the Apache httpd.conf file—I changed the IP Address on which Apache listened. I started Apache and everything worked as it was supposed to with no other changes required. This all took about ten minutes and is an excellent example of true portability.

Had those programs been compiled, I would have had to recompile them and remove any hardware specific efficiencies that we might have programmed in.

Portable code runs anywhere so why spend time porting compiled code to other platforms?

Store Data in Flat Text Files

Text is the Universal Interface. It is Portable and it is Open and Discoverable. Special tools are not required to manage Linux configuration files, your favorite text editor will do fine.

All binary formats are by their very nature obscure and proprietary, for example the Windows registry, which is the antithesis of this tenet.

Use Software Leverage

Software leverage means a couple things to me. First, and in the context of this example, it means that by using four command line commands, we are leveraging the work of the programmers who created those commands with over 7,000 lines of C code. That is code that we do not have to create. We are leveraging the efforts of those other, under-appreciated programmers to accomplish the task we have set for ourselves.

Another aspect of software leverage is that good programmers write good code and great programmers borrow good code. Never rewrite code that has already been written.

One of the great advantages of Open Source software at all levels, from the kernel, through the GNU and other utilities, and all the way up to complex applications, is that there is an incredibly large amount of well-written and tested source code out there that can do almost anything you want to do. Just find the pieces you need and include them in your own code. I use my own code over many times in different programs. I also use a lot of code written by other folks when it meets my needs. Either way, it saves me a lot of work and keeps me from having to reinvent perfectly good code.

Take advantage of others’ code and share your own code. Most companies not good at this either internally or externally.

Automate EVERYTHING!

Use Shell Scripts to Increase Leverage and Portability

One shell command can equal thousands or tens of thousands of C instructions. There are a huge number of very powerful shell commands available. There is no need to write and compile low level language programs; it takes far too much time to do so.

Shell scripts are far more portable than C.

Resist the urge to rewrite shell scripts in C or any other compiled language

Avoid Captive User Interfaces

GUI or TUI programs can both be captive interfaces, allowing actions that only the designers decided were acceptable. Captive interfaces also assume user is human which means they cannot be combined with programs using STDIO.

Captive interfaces are generally big and ugly and do not scale well. They provide limited choices and do not leverage software.

Make Every Program a Filter

All programs are filters. Yup, even GUI programs. Programs don’t create data – people do. Computer programs convert data from one form to another. Properly written programs filter data without imprisoning it.

Use STDIO.


The Lesser Tenets

These lesser tenets explain and extend the ten main tenets.

Write programs to work together

Programs should work together. The output of one program should be usable as the input to the next program. This again means that stringing together a series of small programs that each perform a single task well results in a far more flexible working environment than a single, huge, monolithic program that does nothing well. I have seen way too many of those.

Write programs to handle text streams

Text is the universal format for data. Proprietary binary formats are not usable outside the large, proprietary programs that create them. Text can be used by any well-written Linux command line program. In addition, users and administrators can read text and verify that the output is what is to be expected.

The idea of programs working together is totally bound up in the interchangeability of the data involved. Text data can be easily used by almost every Linux utility because they were intentionally written that way.

Universal interface

Always use the universal interface, STDIO. There are three STDIO data streams; STDOUT is output to the display; STDIN is input from the keyboard; STDERR is error messages which are also sent to the display, but which use a different file handle. The powerful functions of piping and redirection work on these STDIO data streams.

Don’t be chatty

Silence is golden. If there is nothing to say a program should say nothing. Many Linux utility programs simply do their work and return to the command line with no messages of any kind. Only if there is an error should most programs have anything to say. There are exceptions, but be sure that it is truly necessary.

Use Regular Expressions (RegEx)

Regular expressions are a very powerful tool that allows for flexible manipulation of text data streams. They make it possible to perform operations on text that are impossible in any other way.

Let someone else do the hard part

Don’t write a new program unless you have to. There are already many programs out there to do 95% of what you need done. If necessary only write a program when you need to handle the other 5%.


The Tao of Linux

Keep It Simple Stupid

References

  1. Eric S. Raymond: The Art of Unix Programming http://www.catb.org/~esr/writings/taoup/html/index.html
  2. Mike Gancarz: Linux and the Unix Philosophy; Digital Press, 2003, ISBN 1-55558-273-7
  3. WkiPedia: http://en.wikipedia.org/wiki/Unix_philosophy
  4. Oregon State University: http://web.engr.oregonstate.edu/~traylor/ece474/lecture_verilog/beamer/linux_philosophy.pdf
  5. Best Unix Quotations: http://www.linfo.org/q_unix.html

There are also other books and articles that document the Linux Way as well as the Unix Way. See the Bibliography for a list.