Last Updated on 11/25/2023 by dboth
In the first article in this series we created our first, very small Bash script. We also explored the reasons for creating shell scripts and why they are the most efficient option for the system administrator rather than compiled programs.
In this article we will begin creation of a Bash script template that can be used as a starting point for other Bash scripts. Our template will ultimately contain a help facility, a GNU licensing statement, a number of simple functions, and some logic to deal with those options as well as others that might be needed for the scripts that will be based on this template.
Why create a template?
Like automation in general, the idea behind creating a template is to be the lazy sysadmin. A template will contain the basic components that we will want in all our scripts. This saves the time required to add those components to every new script and makes it easy to start a new script.
Although it can be tempting to just throw a few Bash commands together into a file and make it executable, that can be counter-productive in the long run. A well-written and well-commented Bash program with a help facility and the capability to accept command line options provides a good starting point for sysadmins who must later maintain the program. That includes the programs we write ourselves and must then maintain.
The requirements
So what are the requirements for this template script? I always create a set of requirements for my scripts, even if it a simple list with only two or three items on it. I have been involved in many projects that either failed completely or failed to meet the needs of the customer, whoever that might have been. This was usually due to the lack of any type of requirements statement or at least a poorly written one.
The requirements for this Bash template are pretty simple.
- Create a template that can be used as the starting point for future Bash programming projects.
- The template should follow standard Bash programming practices.
- A heading section that can be used to describe the function of the program and a change log.
- A licensing statement.
- A section for functions.
- A help function.
- A function to test for whether the user of the program is root.
- A method for evaluating command line options.
The basic structure
A basic Bash script structure has three sections. Bash has no way to delineate each section but the boundaries between sections are implicit.
Of course all scripts should begin with the Sha-Bang (#!) and this one is no different. This should always be the first line of all Bash programs.
The functions section begins after the Sha-Bang. This is the required location for functions in Bash. They must appear before the body of the program. As part of my own need to document everything I place a comment before each function that contains a short description of what it is intended to do. I also include comments inside the functions to provide further elaboration. Short, simple programs may not need functions.
After the function section comes the main part of the program. This can be as simple as a single Bash statement or thousands of lines of code. One of my programs has a little over 200 lines of code, not counting comments. That same program has more than 600 comment lines.
That is all there is. Just three sections compose the structure of a Bash program.
Leading comments
But I always add more than that for various reasons. First, I add a couple sections of comments immediately after the sha-bang. These comment sections are optional but I find them to be very helpful.
The first comment section is the program name and description and a change history. This is a format I learned while working at IBM and it provides a means of documenting the long-term development of the program and any fixes applied to it. This is an important start to documenting your program.
The second comment section is a copyright and license statement. I use the GPL2 or GPL3 and this seems to be a standard statement for programs licensed under the GPL2. If you choose to use a different open source license, that is fine, but I do suggest adding an explicit statement like this to the code in order to eliminate any possible confusion about licensing. I read an interesting article, “The source code is the license,” that helps to explain the reasoning behind this.
Start by renaming hello.sh to BashTemplate.sh then add the code below to get started on our template. Your script should look like this.
#!/bin/bash
################################################################################
# BashTemplate #
# #
# Use this template as the beginning of a new program. Place a short #
# description of the script here. #
# #
# Change History #
# 11/11/2019 David Both Original code. This is a template for creating #
# new Bash shell scripts. #
# Add new history entries as needed. #
# #
# #
################################################################################
################################################################################
################################################################################
# #
# Copyright (C) 2007, 2019 David Both #
# LinuxGeek46@both.org #
# #
# This program is free software; you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation; either version 2 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program; if not, write to the Free Software #
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
# #
################################################################################
################################################################################
################################################################################
echo "Hello World!"Run the revised program to verify that it still works as expected.
About testing
Now is a good time to talk about testing.
“There is always one more bug.”
Lubarsky’s Law of Cybernetic Entomology
Lubarsky – whoever that might be – is correct. We can never find all of the bugs in our code. For every bug I find there always seems to be another that crops up, usually at a very inopportune time.
Testing is not just about programs. It is also about verification that problems – whether caused by hardware, software, or the seemingly endless ways that users can find to break things – that we are supposed to have resolved actually have been. Just as importantly, testing is also about ensuring that the code is easy to use and the interface makes sense to the user.
Following a well-defined process when writing and testing shell scripts can contribute to consistent and high quality results. My process is simple.
- Create a simple test plan.
- Start testing right at the beginning of development
- Perform a final test when the code is complete.
- Move to production and test more.
The test plan
There are lots of different formats for test plans. I have worked with the full range from having it all in my head, to a few notes jotted down on a sheet of paper, to a complex set of forms that required a full description of each test, which functional code it would test, what the test would accomplish, and what the inputs and results should be.
Speaking as a SysAdmin who has been but is not now a tester, I try to take the middle ground. Having at least a short, written test plan will ensure consistency from one test run to the next. How much detail you need depends upon how formal your development and test functions are.
All of the sample test plan documents I found using Internet searches were complex and intended for large organizations with a very formal development and test process. Although those test plans would be good for those with “Test” in their job title, they really do not apply well to System Administrators and our more chaotic and fast time-dependent working conditions. As in most other aspects of our jobs we need to be creative. So here is a short list of things that you would want to consider including in your test plan. Modify it to suit your needs.
- The name and a short description of the software being tested.
- A description software features to be tested.
- The starting conditions for each test.
- The functions to follow for each test.
- A description of the desired outcome for each test.
- Include specific tests designed to test for negative outcomes.
- Tests for how the program handles unexpected input.
- A clear description of what constitutes pass or fail for each test.
- Fuzzy testing, which will be described below.
This short list should give you some ideas for creating your own test plans. For most sysadmins this should be kept simple and fairly informal.
Test early – test often
I always start testing my shell scripts as soon as I complete the first portion that is executable. This is true whether I am writing a short command line program or a script that is an executable file.
I usually start creating new programs with the shell script template. I write the code for the Help function and test it. This is usually a trivial part of the process but it helps me get started and ensures that things in the template are working properly at the outset. At this point it is easy to fix problems with the template portions of the script, or to modify it to meet specific needs that the standard template cannot.
When the template and Help function are working, I move on to creating the body of the program by adding comments to document the programming steps required to meet the program specifications. Now I start adding code to meet the requirements stated in each comment. This code will probably require adding variables which are initialized in that section of the template – which is now becoming our shell script.
This is where testing is more than just entering data and verifying the results. It takes a bit of extra work. Sometimes I add a command that simply prints the intermediate result of the code I just wrote and verify that. Other times, for more complex scripts, I add a -t option for “test mode.” In this case the internal test code is only executed when the -t option is entered at the command line.
Final testing
After the code is complete I go back through a complete test of all the features and functions using known inputs to produce specific outputs. I also test for some random inputs to see if the program can handle unexpected input now that it is complete.
Final testing is intended to verify that the program is functioning essentially as intended now that it is complete. A large part of the final test is to ensure that functions that worked earlier in the development cycle have not been broken by code added or changed later in the cycle.
If you have been testing the script as you added new code to it, there should be no surprises during this final test. Wrong! There are always surprises during final testing. Always. Expect those surprises and be ready to spend some time fixing them. If there were never any bugs discovered during final testing there would be no point in doing a final test, would there.
Testing in Production
Huh – what?
“Not until a program has been in production for at least six months will the most harmful error be discovered.”
Troutman’s Programming Postulates
Yes, testing in production is now considered normal and desirable. Having been a tester myself, this actually does seem reasonable. “But wait! That’s dangerous,” you say. My experience is that it is no more dangerous than extensive and rigorous testing in a dedicated test environment. In some cases there is no choice because there is no test environment – only production.
Sysadmins are no strangers to the need to test new or revised scripts in production. Any time a script is moved into production that becomes the ultimate test. The production environment itself constitutes the most critical part of that test. Nothing that can be dreamed up by testers in a test environment can fully replicate the true production environment.
The allegedly new practice of testing in production is just the recognition of what we sysadmins have known all along. The best test is production – so long as it is not the only test.
Fuzzy testing
This is another of those buzzwords that caused me to roll my eyes when I first heard it. I learned that its essential meaning is simple – have someone bang on the keys until something happens and see how well the program handles it. Fuzzy testing is like the time my son broke the code for a game I was writing for him in less than a minute with his random input. That pretty much ended my attempts to write games for him. But there really is more to it than that.
Most test plans utilize very specific input that generates a specific result or output. Regardless of whether the test is for a positive or negative outcome as success, it is still controlled and the inputs and results are specified and expected, such as a specific error message for a specific failure mode.
Fuzzy testing is about dealing with randomness in all aspects of the test such as starting conditions, very random and unexpected input, random combinations of options selected, low memory, high levels of CPU contention with other programs, multiple instances of the program under test, and any other random conditions that you can think of to be applied to the tests.
I try to do some fuzzy testing right from the beginning. If the Bash script cannot deal with significant randomness in its very early stages then it is unlikely to get better as we add more code. This is a good time to catch these problems and fix them while the code is relatively simple. A bit of fuzzy testing at each stage of completion is also useful in locating problems before they get masked by even more code.
After the code is completed I like to do some more extensive fuzzy testing. Always do some fuzzy testing. I have certainly been surprised by some of the results I have encountered. It is easy to test for the expected things, but users do not usually do the expected things with a script.
Previews of coming attractions
We accomplished a little in the way of creating our template in this article but we mostly talked about testing. This is because testing is a critical part of creating any kind of program. We are not done creating our Bash script template. In the next installment we will add a basic help function along with some code to detect and act on options such as -h.
Series Articles
This list of links contains all of the articles in this series about Bash.
- Learning Bash and How to Program It. An introduction to this series.
- Programming Bash: Getting Started
- Programming Bash: Logical Operators
- Programming Bash: Loops
- Automation With Bash Scripts: Getting Started
- Automation With Bash Scripts: Creating a template
- Automation With Bash Scripts: Bash Program Needs Help
- Automation With Bash Scripts: Initialization and sanity testing