23
Sat, Nov
1 New Articles

The Linux Letter: Don't BASH Your Head Against a Wall Learning QSH!

Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

When I was first exposed to CPF (OS/400's predecessor on the System/38), I was struck by the elegance of its command shell. It was certainly an improvement over that available within my System/34's SSP environment and orders of magnitude better than the old RSTS/E interface on my DEC system.

It has been said many times that OS/400 is a programmer's dream, and I would have to agree with that. Having a command-prompting system that works both at the command line and from within an editor is "sweet." But I have to confess that, once I started working with the Bourne Again Shell (BASH), the default command processor supplied with most of the Linux distributions I've used, I started getting shell envy. There are some things that are a royal pain to do in OS/400 that are trivial exercises in BASH. For example, think about the steps you would take to perform some function over an indeterminate list of objects. You'd have to build the list, using either an OS/400 API or the OUTPUT(*OUTFILE) function of an OS/400 command, then iterate over that list to perform your function on each item. Within BASH, you can easily perform such a task with a "one-liner." We'll look at some examples that demonstrate this function later.

Fortunately, IBM has seen fit to bring some of the functionality of BASH to OS/400 via its QSHELL interface. And if you start to find yourself limited by QSH, you can always install PASE (available as a PRPQ on OS/400 V4R5 but included with V5R1 and later). Spending time in QSHELL (STRQSH or QSH) will allow you to be more comfortable working within a BASH shell and vice versa.

Throughout this article, I'll be referring to BASH, but much of what I say is applicable to QSHELL. Furthermore, wherever you see the word "Linux," you can substitute any of the Unix-like OSes.

A Little Design Philosophy

The biggest culture shock you'll receive when moving from OS/400 to Linux is how files are handled. In OS/400, everything tends toward fixed-length records. Even the green-screen interface is record-oriented--the system sends a record containing the screen format and application data to the terminal; it's displayed to the user for her perusal and possible update; then, when she hits Enter or a command key, a record is sent back to the application program for further processing.

On the other hand, Linux and all the other Unix-like OSes see files as streams of data. Besides the obvious objects that we would recognize as files, such as Java source programs or class files, Linux treats virtually all objects as files. This includes devices, such as the floppy driver or audio device. Such a design requires a little time to absorb, since it's so different from the rigid (not necessarily a bad thing) structure that OS/400 applies to its objects. For example, in Linux you can treat the floppy drive as a file-structured device with a File Allocation Table (FAT) or Virtual File Allocation Table (VFAT) format and use familiar commands to copy files to it. Or you can simply treat it as a piece of magnetic media that holds approximately 1.5 MB and issue a command such as cp myfile /dev/fd0, which would copy the file myfile to the diskette. Subsequent directory commands applied to that particular diskette would result in error messages, since the FAT structure would be destroyed. But the command cp /dev/fd0 myfile would produce the expected results.

Linux will automatically create the following implicit files for each process within your job stream: stdin, stdout and stderr. Most standard Linux commands will, by default, obtain input from stdin, write their output to stdout, and dump error messages to stderr.

These facts would be undeniably underwhelming if it weren't for another design feature called "piping." Piping allows you to direct output (stdout) from one command into the input (stdin) of another. Thus, you can chain commands together to create your desired results. Piping isn't a facility that's unique to Linux. It's available in any of the Windows flavors, and if you care to do some archaeology within your technical library, you'll see that even DOS has the facility (although cruder in implementation).

More Design Philosophy

In a popular episode of M*A*S*H, Charles Emerson Winchester III was admonished to "hurry up" with his surgery. His reply: "I do one thing at a time. I do it very well. And then I move on." The GNU tools that are included with Linux were designed to do one thing at a time and do it very well. Each of these tools has a very specific purpose. Yet, when chained together via the pipe facility, they make an extremely flexible and powerful tool.

Let's look at a simple example using the file that contains user information on a Linux box. The file /etc/passwd contains, among other information, user names that we'd like to extract. Figure 1 shows the results of issuing the cat command on the file.

[klinebl@laptop1 current]$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
news:x:9:13:news:/var/spool/news:
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin
mailnull:x:47:47::/var/spool/mqueue:/dev/null
rpm:x:37:37::/var/lib/rpm:/bin/bash
ntp:x:38:38::/etc/ntp:/sbin/nologin
rpc:x:32:32:Portmapper RPC user:/:/sbin/nologin
xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false
gdm:x:42:42::/var/gdm:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
nscd:x:28:28:NSCD Daemon:/:/bin/false
ident:x:98:98:pident user:/:/sbin/nologin
radvd:x:75:75:radvd user:/:/bin/false
postgres:x:26:26:PostgreSQL Server:/var/lib/pgsql:/bin/bash
apache:x:48:48:Apache:/var/www:/bin/false
squid:x:23:23::/var/spool/squid:/dev/null
pcap:x:77:77::/var/arpwatch:/sbin/nologin
klinebl:x:500:500::/home/klinebl:/bin/bash
tomcat4:x:91:91:Tomcat4:/var/tomcat4:/bin/bash

Figure 1: The Linux user password file looks like this in its raw form.

Note that the fields are separated by the colon character (:). The cat command's purpose is to copy one or more files whose names have been passed as a parameter to stdout and nothing more. The results of this command are so far unremarkable because we're interested only in the user name, which is in the first field. So let's use the filter cut to extract the user name. Figure 2 shows the first results we obtained from cat, but this time we piped the output of cat to the input of the filter cut, the purpose of which is to "remove sections from each line of files" (from the manual page for cut). The parameter -f1 means field one, and the -d: parameter means that fields are separated by colons. The vertical bar character (|) is used to denote piping from one filter to another.

[klinebl@laptop1 current]$ cat /etc/passwd | cut -f1 -d:
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
news
uucp
operator
games
gopher
ftp
nobody
vcsa
mailnull
rpm
ntp
rpc
xfs
gdm
rpcuser
nfsnobody
nscd
ident
radvd
postgres
apache
squid
pcap
klinebl
tomcat4

Figure 2: Once filtered through cut, the output of our command string contains only user names but is still in sorted order.

What if you want the results in sorted order? No problem. We'll just pipe the output from cut to the input of another filter, sort. This filter's name is more intuitive; the manual page states: "sort lines of text files." Figure 3 shows the sorted results. You can see how to chain together simple programs, which do very simple and specific tasks, to create very powerful programs.

[klinebl@laptop1 current]$ cat /etc/passwd | cut -f1 -d: | sort
adm
apache
bin
daemon
ftp
games
gdm
gopher
halt
ident
klinebl
lp
mail
mailnull
news
nfsnobody
nobody
nscd
ntp
operator
pcap
postgres
radvd
root
rpc
rpcuser
rpm
shutdown
squid
sync
tomcat4
uucp
vcsa
xfs

 

Figure 3: That's better! We now have a list of all of the users defined on our system, and in sorted order, too.

 

Earlier, I mentioned that I would give an example of performing a task on an indeterminate list of items. Let's use our sorted list of user names and send each of them a greeting.

Figure 4 shows an example using the BASH for control structure, the basic form of which looks like this:

for name [ in word ] ; do list ; done

 

Here, name represents a variable and [ in word ] represents a list of items. In Figure 4, the commands within the dollar sign/parentheses ($()) are interpreted first. Its output then replaces the commands within the statement, and execution of the for statement is performed on each member of the list.

[klinebl@laptop1 current]$ for u in $(cat /etc/passwd | cut -f1 -d: | sort);do echo Hello, $u!;done
Hello, adm!
Hello, apache!
Hello, bin!
Hello, daemon!
Hello, ftp!
Hello, games!
Hello, gdm!
Hello, gopher!
Hello, halt!
Hello, ident!
Hello, klinebl!
Hello, lp!
Hello, mail!
Hello, mailnull!
Hello, news!
Hello, nfsnobody!
Hello, nobody!
Hello, nscd!
Hello, ntp!
Hello, operator!
Hello, pcap!
Hello, postgres!
Hello, radvd!
Hello, root!
Hello, rpc!
Hello, rpcuser!
Hello, rpm!
Hello, shutdown!
Hello, squid!
Hello, sync!
Hello, tomcat4!
Hello, uucp!
Hello, vcsa!
Hello, xfs!

Figure 4: This shows one method to make use of the list we created in Figure 3.

 

So far, I've only been discussing piping. Another feature, called redirection, allows you to change the definitions of stdin, stdout and stderr. A quick example can be shown by rewriting the command used to produce the results in Figure 3. By typing the code below, we eliminate the need to call cat and, instead, redirect stdin so that the contents of /etc/passwd are sent directly to stdin of the cut command.

cut -f1 -d: < /etc/passwd | sort

The operators for redirecting stdin and stdout are the less-than symbol (<) and the greater-than symbol (>), respectively. Purists will tell you that the latter example is more efficient than the former (and they would be right). But the choice is a mostly matter of taste, since the current machines are so blazingly fast that the additional overhead caused by using the first form will be imperceptible.

Back at the QSHELL Prompt

At this point, you may be wondering how QSHELL would be useful in OS/400. I can explain that in an acronym: IFS (Integrated File System). As much as I love the OS/400 command line QCMD, it seems somewhat anachronistic when dealing with objects in the IFS. I'm so used to working at a BASH command line that most of its commands have been committed to muscle memory. Trying to maneuver around the IFS while in OS/400's shell, using commands like WRKLNK, is downright painful. By issuing the command QSH, I can be back in more familiar territory.

Once the QSH prompt is displayed, you can use the pwd command (print working directory) to see where you are. By default, you should find yourself in the root directory, denoted by the single slash character (/). But you can use OS/400's CHGUSRPRF profile HOMEDIR('/new/home/directory') parameter to change the default directory for any user.

If you issue the command ls, you'll see the contents of the current directory. You can use the cd command (change directory) to move to other subdirectories, just as you would in DOS or Windows, except the forward slash is used instead of the backslash character ().

I've just rattled off three of the commands available to you in QSH. Unfortunately, BASH doesn't have the menu structure of OS/400 (like GO CMDWRK), so you won't be able to use it to learn all of the commands. QSHELL also lacks BASH's apropos command, a facility that provides some of the same information for BASH commands as that available to users in OS/400. Fortunately, IBM does have documentation for QSHELL (available here) that you can use to learn more about QSHELL and its commands and control structures.

Perhaps you still are skeptical about QSHELL because all of your development is in ILE/RPG and you don't find yourself dealing with the IFS very often. That may be true, but you can access your source members via the IFS. Just use this path:

'/QSYS.LIB/yourlibrary.lib/yoursourcefile.file
/yoursourcemember.mbr'

 (NOTE: The above should be all one line when used.)

You'll be able to use any of the QSH commands just as though the file were stored in the native IFS. Thus, the member MYRPG in source file BARRY/QRPGLESRC would be accessed this way:

 '/QSYS.LIB/BARRY.LIB/QRPGLESRC.FILE/MYRPG.MBR'
  

You can test drive this capability by entering the command ls /your/path/to/member/here.

If you still aren't convinced that any of this is interesting, keep in mind that IBM now makes it possible to store RPG source code in the IFS so that you may avail yourself of the newest PC-based editors. Sooner or later, you will find yourself straying onto the IFS's turf--it's just a matter of time. So you may as well get a head start.

Jump into the Stream--The Water Is Fine!

I have given you only the briefest of introductions to BASH and haven't even begun to scratch the surface of the richness of its commands and filters. I do hope that it's enough to pique your interest. You can read more in the documentation for BASH, which is available at the Free Software Foundation's Web Site.

Those of you who are running Linux can immediately begin to experiment with BASH simply by opening a command shell. If you are unfortunate enough to be stuck using a different PC operating system, then all is not lost. There is a BASH environment available free that will load right up on your Windows box. If you point your browser to Red Hat's site, you'll be able to download a product titled Cygwin, which is not only a port of the BASH shell to Windows but also most of the GNU tools. This will allow you to learn BASH, as well as all of those other tools and features, in preparation for your eventual (perhaps even inevitable) migration from Windows to Linux--or from the QCMD environment to the QSH environment.

And finally, I'd like to wish everyone a happy and safe holiday season! See you next year.

Barry L. Kline is a consultant and has been developing software on various DEC and IBM midrange platforms for over 20 years. Barry discovered Linux back in the days when it was necessary to download diskette images and source code from the Internet. Since then, he has installed Linux on hundreds of machines, where it functions as servers and workstations in iSeries and Windows networks. He recently co-authored the book Understanding Linux Web Hosting with Don Denoncourt. Barry can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..

Barry Kline 0

Barry L. Kline is a consultant and has been developing software on various DEC and IBM midrange platforms since the early 1980s. Barry discovered Linux back in the days when it was necessary to download diskette images and source code from the Internet. Since then, he has installed Linux on hundreds of machines, where it functions as servers and workstations in iSeries and Windows networks. He co-authored the book Understanding Web Hosting on Linux with Don Denoncourt. Barry can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: