unicorn-utterances/content/blog/shell-scripting-basics/index.md

title, description, published, authors, tags, attached, license

title	description	published	authors	tags	attached	license
Getting Started With Shell Scripts: The Basics	An introduction to the amazing possibilities offered by shell scripting.	2019-09-26T05:12:03.284Z	adueppen fennifith	shell linux		cc-by-4

Intro

Since the days of the first digital computers, interacting with them has been a rather fundamental aspect. Even though we've advanced far past the humble days of thousands of switches, massive control panels, and hundreds of punchcards into an era of smooth, sleek graphical user interfaces (GUIs), the venerable command line interface (CLI) still remains quite useful.

On Unix and Unix-like OSs, such as Linux and macOS, a program known as a shell is responsible for providing the CLI. The most common one today is bash, and it can be found on the majority of Unix-like systems. It supports a shell scripting language dating back to the Thompson shell from 1971, although much of the scripting functionality available in modern shells comes from the Bourne shell.

Compared to many other scripting languages, shell scripting is a bit odd because all of the keywords found in it (such as for, if, while, etc) are actually commands built into the shell, and not special keywords being interpreted separately from the other commands in the script. This means that it's possible to use these commands anywhere, even outside of a script. For years, I didn't even know that common commands like echo were actually being run directly in the shell and not with a separate program!

Before we start, I'd like to say that this post is geared more towards people who already have some prior programming experience, and some existing Linux/Unix command-line experience helps as well.

Setup

In order to follow this post, you'll want to make sure that you have access to a Linux shell of some sort. Thankfully, this is easier than ever to find nowadays. Everything should work the same whether it's running in a Linux VM, macOS, WSL, Termux, or Crostini. All of these include bash by default, but to make sure, run the command which bash. This will print out the location of bash on your system, and as long as the message gives some sort of location, such as /bin/bash, you should be fine.

Although not strictly necessary, you'll probably want to make sure you have a text editor with support for shell script syntax highlighting installed. nano or micro are good options on the command-line, especially if you're new to CLI text editors. With that out of the way, let's get to the actual shell scripting part of this post!

Basic Scripting

---

A Simple Example

At its core, a shell script is simply a sequence of commands, one per line - if you've used a bash command line before, you already know a lot of the syntax it uses. Each line of the file is interpreted as a single command, and bash will run each line sequentially until it reaches the end of the file, or it gets to an exit command. If you're ever confused about how part of a script works, you can try running it individually outside of the script to see how it behaves.

Let's start out with a simple example: create a file somewhere on your computer called simple.sh and start editing it. In most text editors, you can combine both steps into one by running {editor} simple.sh - nano simple.sh, for example. All you need to put into the file is the text pwd. This is a standard Linux/Unix command that prints out the directory you're currently in. If you save and exit your editor and then run bash simple.sh, it should print out your current directory. In my case, it looks like this:

$ bash pwd.sh
/home/adueppen/scripts

Congratulations! You've just written your first shell script! Sure, it just does the same thing that running pwd in the terminal normally would do, but the real power of shell scripting comes in when using these standard commands along with the scripting features in order to add things like interactivity, loops, and conditions. Now, let's move on to some of those more advanced features.

Cowsay, Part 1: The if Command

Conditional execution is one of the most important parts of any programming language. If you couldn't choose whether or not to execute something, things would be ...difficult, to say the least. Thankfully, bash includes if as a built-in command. Note that this is technically different from a typical programming language's if statement, and that's why I've been referring to it as the "if command". In practice, however, it functions basically the same way.

A common way that conditions are used in shell scripts is to check if a particular program is installed before running it. Here's a simple example of it in use:

if command -v cowsay > /dev/null
then
  cowsay "Hello, world!"
fi

Here, an if construct is used with the built-in command command in order to check if the cowsay command exists. It's generally not installed by default (even if it perhaps should be), so we can't assume that it's going to be there. The > /dev/null part is used to throw away the output of command since we don't actually need to know where cowsay is, just that it exists. If it's there, we run cowsay with the argument "Hello, world!", which prints a funky looking cow in the shell.

Although the use of redirection (>) might seem a bit confusing now, we'll get into that later. Right now, if you run this script, it'll either print out ASCII art of a cow saying "Hello, world!", or ...nothing. That's not exactly ideal, so let's add an alternative case for when cowsay isn't available:

if command -v cowsay > /dev/null
then
  cowsay "Hello, world!"
else
  echo "Hello, world! (PS: install cowsay for more fun!)"
fi

Now our script prints out "Hello, world!" even if cowsay isn't installed. If you don't have cowsay on your computer, try installing it to see the output change!

Cowsay, Part 2: User Input

Let's make this script a little more interactive - what about asking the user what they want the cow to say? One way to do that is to use the read command. Here's an excerpt from the command's help text, which can be accessed by running read --help in your command line.

read: read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name ...]

Reads a single line from the standard input, or from file descriptor FD if the -u option is supplied. The line is split into fields as with word splitting, and the first word is assigned to the first NAME, the second word to the second NAME, and so on, with any leftover words assigned to the last NAME. Only the characters found in $IFS are recognized as word delimiters.

-p prompt - output the string PROMPT without a trailing newline before attempting to read

A lot of this isn't useful to us, but we can extract a bit of helpful information:

• The (simplified) syntax of the command we want to run is read -p [prompt] [name]
• read will allow the user to enter a single line, and store its result in whatever variable we specify as [name]
• By passing -p [prompt], we tell the command to output the prompt text before accepting any input.

Now, let's try putting this together - if we want to ask the user a question, we could write something like this:

read -p "What should the cow say? " input

Running this will prompt the user for a line of text and store its result in a variable named input - but how do we use that variable in our code?

The syntax for accessing variables in bash is to prefix their name with a dollar sign ($). With this, variables can be embedded in nearly any bash command or argument - they will simply be replaced with the content of the variable when the command is run. With this knowledge, let's rewrite our example of the if command from before...

read -p "What should the cow say? " input

if command -v cowsay > /dev/null
then
  cowsay "$input"
else
  echo "The cow says $input (PS: install cowsay for more fun!)"
fi

Optionally, the variable can be formatted as ${input}, with curly brackets around the name - this is sometimes useful to make it distinctly separate from the rest of the string, if it's surrounded by other text.

Try running this! Your script should now ask you to enter a line of text, which will then be passed to the cowsay command, generating a fun ASCII cow that says whatever you want.

Cowsay, Part 3: Command Substitution

read obviously works a little bit differently from most bash commands, in a "pass-by-reference" design - this isn't unusual, and we'll cover how this works in a future post, but there are more common ways to assign a variable to the result of a command. Namely, using the variable=value syntax...

For a hint to why read needs to do this, try running read -p "Type something: " in your command line. You'll notice that, after prompting you for input, it doesn't write anything else to the console. Most other commands (such as pwd) will print their results to the "standard output" - which this next feature makes use of.

Let's say that we want our cow to greet the user at the start of our script. The whoami command is a fairly standard Linux tool that prints the name of the current user - running this in your command line, you should see that it prints your username. We now know the command we want to run to get the information we need, but how can we use this in our script?

For this, we need to capture the output of the command and store it in a variable. Bash allows us to do that using the "command substitution" construct - by enclosing our command in $(...). All we need to do is assign that to a variable, then use that variable to template it into our greeting.

username=$(whoami)
echo "Hello, $username!"

If you guessed that we could shorten this to a single line, you'd be right! Just like how we can template variables in a double-quoted string, command substitution can be templated as well - making our command echo "Hello, $(whoami)!".

Now that we've got this working, all we have to do is run cowsay instead of echo - and we have a very creepy cow that somehow knows what my name is. Oh no. I could have nightmares about this.

Err, let's forget about that for now and add this to the start of our program - enclosing it in the if statement we wrote before, to make sure that cowsay actually exists before using it.

if command -v cowsay > /dev/null
then
  cowsay "Hello, $(whoami)!"
else
  echo "Hello, $(whoami)! You should really install cowsay. It's a lot of fun."
fi

read -p "What should the cow say? " input

if command -v cowsay > /dev/null
then
  cowsay "$input"
else
  echo "The cow says $input (PS: install cowsay for more fun!)"
fi

Cowsay, Part 4: Test Constructs

Now that we've added a couple features to our program, we have the ability to get our cow to say some pretty crazy things. I've gotten mine to say "woof," for example - a noise that no real cow should ever be making. Perhaps we should add an extra case in this script to prevent our cow from making such a terrifying sound? In order to accomplish this, we need to learn about test constructs, and how the if command really works.

When any shell command is executed, it implicitly sets a variable named $?, called its "exit code". This is always an integer, and can be seen as a sort of "return value" in bash. Normally, most exit codes should be 0 - by Unix convention, this indicates that the command was a success. Some commands do fail, though - and their exit code can be used to communicate that information to the program that started it.

As an example of this, the command command has the potential to fail if the argument it is given doesn't exist. Running command thiscommanddoesntexist followed by echo $? should print the number 127 in your shell - while running command echo followed by echo $? will print 0. This is how the if statement evaluates its condition - if it returns an exit code of zero, it is interpreted as a success.

With this in mind, we're going to introduce "test constructs" - a shell syntax that evaluates its arguments as a comparison, and returns an exit status indicative of the result (0 for true, 1 for false). Similarly to how if is secretly a command of its own, test constructs are also a command... which is named... "left square bracket" ([). No, I'm not joking.

Here, we want to compare two variables for equality - we want to check whether our $input variable is equal to "woof". The "left square bracket" equivalent of this should then be [ "$input" == "woof" ]. If the comparison is a success (meaning that $input is "woof"), the exit status will be 0, which should be interpreted by an if command as "true." Inside our if statement, we can echo a message to the user, then use exit 1 to terminate the script and indicate a failure - if another script wanted to determine the result of ours, it could use this exit code to do so.

if command -v cowsay > /dev/null
then
  cowsay "Hello, $(whoami)!"
else
  echo "Hello, $(whoami)! You should really install cowsay. It's a lot of fun."
fi

read -p "What should the cow say? " input

if [ "$input" == "woof" ]
then
  echo "Your cow sounds like it has a cold! Take it to the vet."
  exit 1
fi

if command -v cowsay > /dev/null
then
  cowsay "$input"
else
  echo "The cow says $input (PS: install cowsay for more fun!)"
fi

One important thing to note about these test constructs: the space between the square brackets and the condition is mandatory. The [ command is interpreted as, well, a command - leaving out that space between [ and "$input" would tell bash to look for a program named ["$input" instead; the same thing would happen if you were to write exit1 instead of exit 1.

Another thing we didn't quite cover here - the "if" construct also supports elif. It can be used the same as an else case, in the form of elif [condition]; then ....

Executable Scripts

More Features

By now, we've probably exhausted our poor cow of examples, and you've learnt a lot of scripting concepts from it. For this section, we're going to forego the theatrics and just list a bunch of functionality for you to try out on your own; much of this is very similar to what we've already covered and shouldn't be difficult to pick up by referencing what you've already written.

Looping Control Structures

Bash also includes a few commands for looping or repeatedly performing a task: for, while, and until.

while loops have a similar syntax to the if command, with a few key differences...

i=5
while [ $i > 0 ]
do
    i=$(expr $i - 1)
done