Get more familiar with the Linux command line and add to your Linux skills.

The Linux shell, commands, and redirection can be intimidating to the Linux novice. The useful commands, tips, and tricks in this TechTip will help you learn to use the Linux command line with confidence.

The Shell

What's a shell? Think of the shell as nothing more than the interpreter between your user (or administrator) and the Linux kernel. There are many Linux shells, but some of the most popular ones are bash (Bourne-again shell GNU), csh (C shell BSD), ksh (Korn shell Bell labs), and zsh (extension of the Korn shell). Linux is improving in allowing the use of GUIs for administration purposes, but if you're going to administer Linux systems, then you absolutely must be comfortable at the command line from time to time.

Finding Commands and Utilities

Probably the single most frustrating aspect of the Linux command line is not knowing what tools and commands are at your disposal. Finding the tools you need to administer a Linux system is key to learning how to use the tools. Combine this with no real standard naming convention, and you can quickly get upset when you can't locate a command. Fortunately, there are a few utilities available that build a database of what does what on your system.

The first utility is the apropos command. Running apropos allows you to search on part of a name--or better, a description of what a command does. This is perfect for searching for a command when you don't know the name. In the following example, searching for a string of "secure copy" yields the programs containing the string.

# apropos "secure copy"

scp  (1)  - secure copy (remote file copy program)

If you search a string without the quotations, the results will return anything that contains the word "secure" or "copy." If you search with the string in quotations, the results will return only those items with both "secure" and "copy" in the name or description.

The second useful search command is whatis. The whatis command gives you a description based on a command name. This is useful if you know a Linux command but have forgotten what the command actually does. Since this is useful only if you already know the command name, whatis might not be much help when learning, but it serves its purpose.

# whatis tail

tail                 (1)  - output the last part of files

tail                 (1p)  - copy the last part of a file

Another common tool used for locating tools is whereis. This command allows you to find where a command or program resides on the file system. Why do you need to know this? When you start any kind of scripting, it's good practice to specify the full path to the commands being used.

# whereis telnet

telnet: /usr/bin/telnet /usr/share/man/man1/telnet.1.gz

The last, and most commonly used command for me, is the man command. Virtually every Linux command comes with a manual file that specifies how to use the command. In the above example, the whereis output displayed that the telnet executable resides in /usr/bin/ and has a manual page located at /usr/share/man/man1/telnet.1.gz. When using the man command, you need only specify the command name, not the full path to the manual file.

# man telnet

When you're done reading the manual file, you can hit the "q" key to quit and return to the shell. For many system administrators, these commands might be second nature. If you don't know these basic tools for locating commands on a Linux system, though, it can be quite cumbersome when starting out on the command line.

Redirection

Another simple, yet often unexplained, concept of Linux is redirection. Redirection is easy, and it's used every day when working with files and programs. Here's a list of redirection symbols and their uses.

• > Redirect output
• < Redirect input
• >> Redirect output appending
• | Pipe output to other commands
• >& Redirect both standard output and error
• 2> Redirect error
• /dev/null Redirect output not to be saved

Redirection can be commonly used to direct output to text files or to other commands. For instance, to get a directory listing of space usage of files and folders in a specific directory, you can use the du (disk usage) command. You can then redirect the output of the program to a text file for a text report you can use. Afterward, you can use the cat (concatenate) or vim (Visual Editor Improved) command to look at or edit the list created.

# du --max-depth=1 -h /home/max/ > directory_usage.txt

# cat directory_usage.txt

# vim directory_usage.txt

To sort the list before outputting it to the text file, you can redirect the output of du to the sort command by using the numeric sort switch.

#  du --max-depth=1 -h /home/max | sort -n > directory_usage.txt

Redirecting input works the same way. Create a text file with your favorite text editor and enter random numbers (not in order). Then run the sort command but read the input in from the text file you created. It will sort the lines by numeric standard.

# vim numbers.txt

10

7

5

3

6

# sort -n < numbers.txt

3

5

6

7

10

All other redirection symbols listed above work in the same fashion, so experiment with piping and redirecting output to get used to working with redirection. Just be careful of piping and redirecting output to the rm (remove) command. On the command line, Linux will not ask you for verification before deleting items specified. Trust me, I know. Sending output to /dev/null sends it to nowhere (often referred to as the "bit bucket"), which makes it perfect when you want no logging or notification from a command.

Brace and Tilde Expansion

The Linux shell allows you to do some time-saving expansion, making a single command do two things so that you don't have to type out full commands and file names. With brace expansion, you could easily rename a file without having to type the file name twice. While this doesn't seem like much, it allows for easy renaming of multiple files within scripts.

# touch testing.txt

# mv testing.{.txt,.html}

On the command line, the tilde (~) is prefixed to be your user's home directory. When you type the tilde and hit Enter, the following is displayed, showing your home path as a directory.

# ~

bash: /home/max: is a directory

Again, this is a simple example, but it can save the time you'd spend typing out full path names. For example, suppose you're in a directory completely unrelated in path to one you want to change to. Type out the tilde prefix with where you want to change to. By doing so, in the example below, /home/max will be prefixed to everything.

# cd /home/max/downloads/isos

# cd ~/pics

In this example, ~/pics actually means /home/max/pics, which allows for easy movement within the file system. Also, this allows for easy exports of paths using the export command. The default bash export path on my system lists the directories the shell uses for paths to executables. By creating a directory and then using the export function to add this to the list, you are able to add your own paths for the shell to be aware of.

# mkdir /home/max/programs

# echo $PATH

/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/max/bin

# export PATH="$PATH:~/programs"

Now, rerun the echo command, and you'll see that /home/max/programs is now in the shell's search path for executables.

# echo $PATH

/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/max/bin:/home/max/sbin:~/programs

For more documentation on paths, read the PATH HOWTO.

Extracting Output

Using redirection and pipe commands is great, but what if you wanted to do something with some data like extracting several specific things at the same time? There are plenty of tools at your disposal to do exactly that. One such command is awk, a pattern-scanning processing command. Create a simple text file in your favorite editor and create some entries like first name, last name, hire date, and department.

# vim data.txt

John Doe 01/01/60 IT

Jane Doe 01/01/70 HR

Now use the cat command to print the file content, but pipe the command through awk to extract all items.

# cat data.txt | awk '{print $1,$2,$3,$4}'

John Doe 01/01/60 IT

Jane Doe 01/02/70 HR

You should notice that each number corresponds to a match in between spaces. This allows you to extract only certain pieces of information. For example, if you want only the last name of a person and the department he works in, you can specify only those items.

# cat data.txt | awk '{print $2,$4}'

Doe HR

Doe IT

A similar command is the head command, which allows you to output the first parts of a file. Using the -c switch, you can print the first number of bytes specified or, using the -n switch, the first number of lines specified. If you want only the first five characters, issue the following command:

# head -c 5 data.txt

John

There are literally dozens of ways to extract data, and again, although these examples seem simple, the power lies in their ability to be easily scripted in programs and shell scripts.

Dangerous Linux Commands

Great options and great flexibility come with a great tradeoff. Like I mentioned earlier with watching what commands you pipe to the remove command, you need to be cautious. Here are a few examples of some extremely dangerous commands to type in a root shell of a Linux machine.

Do not type the following command or any other command and specify it to output to a live file system. This will destroy a live file system residing on either /dev/sda or /dev/hda. If you happen to be on the running file system, you've just ruined your installation.

# mkfs.ext3 > /dev/{s,h}da

# any-command > /dev/{s,h}da

Everyone wants to know what happens when you remove the root file system under Linux. The answer is exactly what you expect: it starts removing everything. I did this on a test installation just to see the results. It's not pretty, and it renders a useless desktop or server.

# rm -rf /

Moving files is common, unless you're moving things to the bit bucket of /dev/null. Be careful what you send to /dev/null, because unless you have a backup, the black hole will never give your data back.

# mv ~/* /dev/null

# mv /home/max /dev/null

One of the most-used Linux commands is the dd (data definition) tool. This tool allows you to copy, block by block, devices or files and pipe them to anywhere. It's great for copying data across disks, but be careful with where you send your data. Do not type the following if the output is to a live file system.

# dd if=file_or_data of=/dev/{s,h}da

Be very careful when automatically executing shell scripts after downloading them, especially if you haven't had the opportunity to view them. A shell script could contain anything malicious that the author wanted to execute.

# wget http://path/to/shell_script -O- | sh

These are just a few examples of things to be cautious about. Be careful typing out commands when you don't know what they are going to do. This is why you should always read the manual of a command before testing it out on a live system, especially when starting out with Linux.

Start Playing

"The Linux command line is hard!" I hear this all the time, and to some extent, I agree. But with some testing, reading, and the ability to locate commands and tools on your systems, you can start feeling more comfortable at a command prompt. Although I do rely on many useful GUI tools, the command line is where systems administrators of Linux machines live. It's impossible to administer Linux systems without having some navigation skills at the command line.

The shell is a powerful beast, but use some of the basic examples I've described and start trying the more-advanced features of Linux. If you're uncomfortable doing so on a live system, set up a virtual machine in Xen or VMware or perform a Linux install on some old hardware, where you won't hurt live data. Don't be afraid.