Arguments, Lists & Loops

A Practical Introduction

12 Command Line Arguments

When you run a script you can pass values to it on the command line, just as you pass values to commands like ls or grep. These values are called arguments, and inside the script the shell makes them available automatically in a set of numbered variables.

12.1 $1, $2, $3 …

The first argument on the command line is stored in $1, the second in $2, the third in $3, and so on. You can use their value in the script exactly like any other variable.

Create a script called greet2 and make it executable with chmod +x greet2:

Then run it with an argument:

You can use as many arguments as you need. Here is a script that takes a first name and a last name:

If a user passes fewer arguments than the script expects, the missing variables are simply empty. If they pass more, the extras are ignored — unless you use them.

There’s a problem with arguments beyond nine. For example it can’t tell whether $10 is $1 followed by 0 or really the tenth argument. To avoid the ambiguity you should put the number in {}, ${10}, ${11} and so on. This is rarely needed in practice.

12.2 $0 — The Script Name

$0 is a special case. It contains the name of the script itself, exactly as it was typed on the command line. This is most useful for printing error messages, because it lets the message tell the user which script the error came from.

When the file is not found, the error message includes the script name from $0. This is a common Unix convention and you will see it in the error messages from standard commands too.

13 Lists

A list in shell scripting is simply a sequence of values separated by whitespace. There is no special syntax for creating one. Any sequence of words is already a list. This is a fundamental idea in shell scripting, and once you understand it, a lot of other things fall into place.

13.1 The Arguments as a List: $@

All the arguments passed to a script can be referred to as a list using $@. You can echo them all in one go:

$@ expands to all the arguments, in order, as separate words. Think of it as shorthand for $1 $2 $3 … however many there are.

13.2 Counting the Arguments: $#

The shell also keeps a count of how many arguments were passed, in the variable $#. This is useful when your script needs a specific number of arguments and you want to check before proceeding:

14 for Loops

A for loop processes a list of values one at a time. For each value in the list, the shell runs the body of the loop with a variable set to that value. When the list is exhausted, the loop ends.

14.1 A Simple Example

The clearest way to see how a for loop works is with a literal list of values:

The for loop repeats everything between do and done once for every item in the list. Each time around the loop the name variable is loaded with the next item in the list. Anything following “in” is considered a list, with each item separated from the next with whitespace.

In this example, name is called the “loop variable”. You can give it any name you like, and it helps if you pick something meaningful!

You can store a list in a variable, and this is very common.

14.2 Looping Over the Script’s Arguments

Rather than a literal list, you can loop over the arguments passed to the script by using $@ as the list. This is one of the most common uses of a for loop:

Each argument becomes the value of filename in turn. The if block inside the loop runs separately for each one.

15 Building Lists in Practice

Literal lists of values are useful for understanding how loops work, but in real scripts the list usually comes from somewhere: the output of a command, or the lines in a file.

15.1 A List from a Command

When you use command substitution as the list in a for loop, the shell runs the command and splits its output on whitespace to produce the list. Each word of output becomes one value.

A simple example — loop over the users currently logged in, as reported by who:

The who command lists logged-in users, one per line. cut -d’ ‘ -f1 extracts just the username from each line. The shell splits the resulting list of names on whitespace and the loop processes them one by one.

Another common example — process every file in a directory:

15.2 A List from a File

You can also build a list by reading the lines of a file. The standard way to do this in ksh is to redirect the file into a while read loop, but a simple approach that works well for straightforward cases is to use cat inside command substitution:

Where the file serverlist might contain:

Each line of the file becomes one value in the list, and the loop body runs once for each server. This pattern — keeping a list of names in a plain text file and looping over it — is extremely common in Unix system administration scripts.

15.3 Combining It All

Here is a more complete script that brings together arguments, loops, and a command-generated list. It takes a directory as an argument and reports the line count of every file in it:

16 Exercises

Work through the following exercises. Write each one as a script file, make it executable with chmod +x, and test it with a variety of inputs.

Section I — Arguments

Section J — Lists and Loops

End of exercises — good luck!

17 The Shebang Line

You might have noticed the first line of shell scripts often look something like this:

This is known as a shebang, a sort of portmanteau as hash-bang. Why bang? That’s what some Americans call an exclamation mark, at least informally!!!

If the shell reads what it thinks is a program (because the X bit is set) it looks at the first two bytes, and if they’re a “#!” it knows that it’s some kind of interpreted language, and the remainder of the line is the language interpreter need to run the rest of it.

If you have been running the scripts in this chapter, they have been working because the shell that is running them is already ksh. But that is not something you can rely on. When someone else runs your script, or when it is run automatically by the system, it may be handed to a different shell entirely — and different shells have small but significant differences in how they handle things.

These differences are sometimes called bashisms (or kshisms, or any other shell name followed by -isms). A bashism is a feature or syntax that works in bash but is not part of the POSIX standard and will not work in other shells. The same idea applies in reverse: a script written for ksh may use syntax that bash or sh does not understand. If your script relies on a specific shell, you should say so explicitly.

All AIX systems support ksh as the default shell, so to make sure your script runs on every AIX machine, write it for ksh.

A complete script with a shebang now looks like this:

The shebang line begins with #, which is also the start of comment character, so the shell also treats it as a comment.

You can also use #!/bin/sh on pretty much all scripts. This is the path to the standard Bourne Shell, and on systems with newer shells there is nearly always a Bourne-compatible shell in this location. If you run the bash shell when it’s been renamed plain sh will normally run in compatible mode.

18 Exit Values

Every script, like every command, exits with a numeric value that tells the caller whether it succeeded or failed. You have already used this idea when writing if statements. The exit value of the condition command is exactly what “if” responds to. Your own scripts are no different.

18.1 The Default Exit Value

If you do not do anything special, your script exits with the exit value of the last command it ran. If that command succeeded, the script exits 0. If it failed, the script exits with whatever non-zero value that command returned.

Here grep finds a match, exits 0, and so the script exits 0. If grep had found nothing, it would have exited 1, and so would the script.

18.2 Setting the Exit Value with exit

You can set the exit value explicitly using the exit command, followed by the number you want to return. This is the right thing to do when your script has finished its work and you want to be clear about what it is reporting back:

The convention is always the same: 0 means success, anything else means failure. The specific non-zero value you choose can carry meaning. Many commands use different non-zero values to indicate different kinds of failure, and you can check the man page to find out what they mean. For your own scripts, 1 is a reasonable general-purpose failure value.

18.3 Exiting Early

You can place exit anywhere in the script, not just at the end. When the shell reaches an exit statement it stops immediately, regardless of how many lines are left. This is useful for bailing out early when something goes wrong:

Checking for problems at the top of the script and exiting early is a good habit. It means that by the time you reach the main work of the script, you already know the inputs are valid. Each guard clause checks one thing, reports a clear error, and exits with a non-zero value so that anything calling your script can detect the failure.

18.4 Checking a Script’s Exit Value

When you call one script from another, or from the command line, you can check its exit value with $? just as you would for any other command:

Or more directly, use the script itself as the condition of an if, since if is already checking the exit value:

This is exactly the same mechanism you have been using with test and grep. Any command, including your own scripts, can be used as the condition of an if.

Reading Input, Conditionals & Cutting Fields

5 Writing a Script File

So far all the examples have been single commands typed at the prompt. Once you have more than one or two lines, or want to run the same sequence of commands again, you need a script file. A script is simply a plain text file containing shell commands, one per line. The shell reads them in order executes each one in turn unless it encounters instructions to change the order, exactly as if you had typed them yourself.

5.1 Creating a Script with vi

Use vi to create the file. Give it a meaningful name — there is no need for a file extension. To create a script called greet, run:

vi opens in command mode. Press i to enter insert mode, then type your commands. When you are finished, press Esc to return to command mode, then type :x and press Enter to write the file and quit.

5.2 Making the Script Executable

A newly created file has no execute permission. Use chmod +x to add it:

You only need to do this once. After that, you can run the script by typing its name preceded by ./ to tell the shell to look in the current directory:

5.3 A First Script

Here is a simple script that asks for a name, works out the day of the week, and prints a greeting. It uses only commands covered in earlier sections.

Note that if the shell encounters a # character it ignores the remainder of the line. This allows you to enter comments. If you start a line with a # the whole line is ignored. And blank lines do nothing. Using comments and blank lines will make your script easier to understand.

Create the file with vi greet, type the following, then save and quit:

LOGNAME is a system variable that holds the login name of the user. We’re copying it to a local variable here, just so it looks neater.

The date command accepts a format string beginning with +. The %A specifier expands to the full name of the current day of the week. The result is captured into the variable dow using command substitution, then passed to echo along with the name the user entered.

7 Reading User Input

The read command reads one line from standard input and stores it in a named variable. On its own, read gives no prompt. It simply waits silently until you type something and press enter, so we’re using the printf command to output a prompt.

7.1 Prompting with printf

Unlinke echo, which always prints a newline after it’s output, printf keeps the cursor on the same line. This looks much neater for the user. You might see some shells that support echo -n to suppress the newline, but this is not consistent across all Unix systems. printf is part of the POSIX standard and behaves identically everywhere, making it the more portable and reliable choice. Most modern shells also support a -p parameter for read, which allows you to specify a prompt. Ksh on AIX doesn’t.

Once stored, the variable behaves identically to one you assigned directly. You can pass it to commands, embed it in strings, use it in comparisons, or store further processing results into it. Here’s a more complex example:

Here the value the user types is passed to wc, and the result is stored in a second variable. The < redirection feeds the file to wc, which then counts the lines and prints the total. This is then assigned to linecount.

7.2 Reading Multiple Variables

You can name more than one variable on a single read command. The shell splits the input on whitespace and assigns each word to the corresponding variable. Any leftover words all go into the last variable named.

8 Conditional Execution: if…then…else…fi

The if statement in the shell works differently from most programming languages. It does not evaluate a boolean expression as in most languages, but it runs a command and responds to whether that command succeeded or failed.

Every command in the shell exits with a numeric return code when it finishes. Generally, 0 means success, and any non-zero value means failure; but you should always check the man page. The if statement treats 0 as true and non-zero as false. This is the same convention used everywhere in Unix, so any command can be used directly as the condition of an if.

8.1 Structure

The if statement must be terminated with a fi, which is if reversed. Reversing the statement that started the block is a common way to end a block in Unix scripts.

8.2 The test Command

Before combining test with if, it is worth understanding what test does on its own. test is a regular command that evaluates a condition and sets its exit code accordingly: it exits 0 if the condition is true, and 1 if the condition is false. When a command completes it sets the exit value to a special variable called $?. You can see what directly by running test and then examining $?

One thing test can do is compare two numbers. The form is to give it the first number, then a comparison operator, then the second number. It’d be great if you could write “test 4 > 3” but unfortunately it would interpret the > as the output redirection character, so instead options like -gt, -lt, -eq, and -ne are used for Greater Than, Less Than, EQual or Not Equal.

The exit code 0 means true (the condition held) and 1 means false (it did not). This is the opposite of most programming languages, but it is consistent with how every Unix command signals success or failure.

test can also compare strings and test whether files exist using other options. For example, -f followed by a filename will return true if the file exists.

The test command supports very many things you might want to check, and you should see the man page for a full list. Here are some common ones.

8.3 Using test with if

Because if runs a command and checks whether it exits 0, and because test exits 0 when its condition is true, the two fit together naturally. You simply put the test command where if expects a command:

A practical example using a file check before processing:

8.4 elif for Multiple Branches

Use elif to add further conditions before the final else:

8.5 Using Any Command as the Condition

Because if simply runs a command and checks its exit code, any command works as the condition — not just test. A very common pattern is to use grep directly:

grep exits 0 when it finds a match and non-zero when it does not. The if responds to that exit code. The output is redirected to /dev/null so grep doesn’t print anything on the screen. Only the exit code matters here.

9 The [ Alias for test

Some people prefer their if statements to have an expression in brackets, as they might be familiar with from other languages. You can do this in ksh because the test command has an alias: [. You can find it at /usr/bin/[ . And yes, [ is a valid Unix filename!

It works exactly the same as test, except it expects a closing ] as its final argument. The ] is not special syntax — it is simply a required parameter that [ checks for and then ignores. The following lines are equivalent:

Notice that there must be a space before the ] — it is a separate argument to [, not part of the syntax. Forgetting the space is a common mistake that produces a confusing error message.

Here is a complete example using [ ] in place of test:

10 Cutting Fields from Command Output

Many commands produce output in columns: ps, df, who, and ls -l are common examples. The cut command extracts specific fields from each line, either by character position or by a named delimiter character. The -d flag sets the delimiter and -f selects which field or fields to extract, numbered from 1.

10.1 Clean Delimiters: /etc/passwd

The cleanest approach is to work with output that already uses a consistent single character as a separator. The /etc/passwd file is a good example — fields are always separated by a single colon, with no padding:

This is extracting fields one and six from the password file – i.e. the username and their home directory. Note the output is also delimited with the delimiter character, in this case :, if you are extracting multiple fields.

Combining cut with grep and command substitution lets you pull a specific value into a variable:

The grep anchors on the start of a line (^) followed by your login name and a colon, so it matches exactly one line. The cut then extracts field 6 from that line. No space normalisation is needed because the delimiter is always a single colon.

10.2 Space-Separated Output and tr

Commands like ps, df, and who separate fields with spaces, but typically use multiple spaces for visual alignment. This is a problem on AIX because cut does not support the -w flag found on BSD systems, which treats any run of whitespace as a single delimiter. When you pass -d’ ‘ to cut on such output, every individual space counts as its own delimiter, making field numbers unpredictable.

The solution is to normalise the whitespace first using the tr command, which “transforms” data. In this case we’re going to use tr with the -s, which squeezes any run of repeated characters down to a single one:

For example:

           38    38    270

This is no good as you don’t know how many spaces there are before the digits.

38 38 270

Here is a practical example extracting available disk space in /var from df output. You can give df a mount-point as a parameter if you don’t want everything listed.

Filesystem   512-blocks      Free     %Used  Iused  %Iused   Mounted on 
/dev/hd1          65536     60512        8%     55      1%   /home

As you can see, we have a header line we don’t want, and the free space is field three. And there are a random number of space between fields. We’ll need to clean this up to get our number.

Putting it together in a script that warns when disk space is low:

10.3 Extracting Specific Fields from ps

Another common use is extracting the process ID from ps output. The -ef flags give a full listing in a consistent format:

11 Exercises

Work through the following exercises at a ksh prompt. Type the commands as shown, observe the output, then experiment with variations.

Section F — Reading Input

Section G — Conditionals

Section H — Cutting Fields

End of exercises — good luck!

Variables & Macro Expansion

1 Variable Assignment

In ksh, a variable stores a value that can be reused later in commands or scripts. Assigning a value to a variable is straightforward: write the variable name, immediately followed by an equals sign and the value — with no spaces around the = sign.

Syntax

Some practical examples:

Variable names may contain letters, digits, and underscores, and must begin with a letter or underscore. By convention, environment variables (those used by the system) are in UPPER_CASE, and variables used in scripts use lower_case, though ksh does not enforce this.

2 Macro Expansion

Macro expansion (also called parameter expansion or variable substitution) is the process by which ksh replaces a variable reference or expression with its current value before executing a command. The echo command is ideal for exploring expansion because it simply prints whatever arguments it receives. You may be familiar with echo from MS-DOS and Windows, where variables are bracketed with a % character. But Unix shell variables are a more mature system.

2.1 The $ Prefix for Variable Names

Prefix a variable name with $ to expand it. The shell sees a $ and knows that what follows is something it needs to substitute. It’s not just used for variables, but this is a common use case.

Basically the shell goes through a line it’s about to run and “fixes up” anything prefixed with a $, and certain other characters, before it runs it.

You can embed variable expansions anywhere in a string or command line:

If your variable expansion runs on into more alphanumeric characters you will need to wrap it in a { and } to avoid ambiguity. Otherwise the { and } are optional.

2.2 Glob Expansion

Glob expansion (also called pathname expansion or filename generation) lets you use wildcard characters in a command to match multiple file or directory names. ksh performs the expansion before passing the resulting list to the command.

Common wildcard characters

Examples using echo to reveal what the shell expands:

2.3 Command Substitution: $() Syntax

Command substitution lets you capture the output of a command and use it as a value — either storing it in a variable or embedding it directly in another command.

The shell runs command in a subshell, collects its standard output (with trailing newlines stripped), and substitutes the result in place of the $(…) expression. Note that the date command prints the current time and date, and the format of it’s output can be modified using a template prefixed with +. We’ll cover this later. The wc -l command simply counts the number of lines in the input.

Backtick alternative

An older syntax uses backtick characters ( ` ) instead of $(…). This is the character on the keyboard found on most keyboards to the left of the digit 1.It’s equivalent in basic use:

The $() form is strongly preferred for several reasons:

• It can be nested cleanly: $(outer $(inner))
• Backticks require backslash-escaping inside nested calls, which quickly becomes unreadable.
• $() syntax is visually clear and consistent with other shell constructs.
• It’s easy to miss a back-tick or confuse it with an apostrophe.

3 Quoting to Prevent Expansion

Sometimes you want ksh to treat special characters — $ * ? and others — literally rather than expanding them. Quoting provides three ways to do this, each with different scope and purpose.

3.1 Backslash ( \ )

A backslash immediately before a character escapes that single character — the shell treats the following character literally instead of interpreting it.

Backslash escape is precise: it affects only the one character that immediately follows it.

3.2 Single Quotes ( ‘ )

Enclosing text in single quotes prevents all expansion within those quotes. Every character is taken literally — the shell performs no substitution, no glob expansion, and no command substitution.

3.3 Double Quotes ( ” )

Double quotes suppress glob expansion and word splitting, but still allow variable expansion ($var and ${var}) and command substitution ($(…)). They are the most commonly used form of quoting in scripts.

Quick comparison

4 Using a Variable in a Command

A practical and very common use of variable expansion is building up path names or directory fragments that are then passed to commands such as ls.

Partial directory name in ls

Suppose you are frequently working inside a versioned project directory and do not want to retype the path each time. Store part of the path in a variable and let the shell expand it for you:

Notice that braces are used around the variable name (i.e. ${project}) to avoid unexpected problems if there was something funny in $project.

You can also store just a partial directory segment — a prefix — and rely on glob expansion to complete it:

5 Exercises

Work through the following exercises at a ksh prompt. Type the commands exactly as shown, observe the output, then experiment with variations to deepen your understanding.

Section A — Variable Assignment and $ Expansion

Section B — Glob Expansion

Section C — Command Substitution

Section D — Quoting

Section E — Variables in Commands

What grep Does

The grep command searches one or more files (or the output of another command) for lines that match a pattern, and prints each matching line to standard output. If no file is specified, grep reads from standard input.

The name stands for globally search for a regular expression and print — a reference to the equivalent operation in the ed line editor.

The most common use is simple: give grep a word or phrase and it returns every line that contains it.

grep error /var/log/syslog

grep ‘disk full’ /var/adm/messages

When more than one file is given, grep prefixes each matching line with the filename so you know where the match came from.

Basic Syntax

grep [flags] pattern [file …]

The pattern is a regular expression (see the section below). Special shell characters such as $ * [ | ^ ( ) \ must be quoted to prevent the shell interpreting them before grep sees them. As a general rule, put the pattern in single quotes:

grep ‘out of memory’ /var/adm/messages

Essential Flags

-v: Invert the match

-v prints every line that does not match the pattern. This is useful for filtering out noise — comment lines, blank lines, or entries you already know about.

grep -v ‘^#’ /etc/services # skip comment lines

grep -v ‘^$’ report.txt # skip blank lines

grep -v DEBUG /var/log/app.log # hide debug entries

-i: Ignore case

-i makes the match case-insensitive. Error, ERROR, and error all match the pattern error when -i is given.

grep -i ‘failed’ /var/adm/messages

grep -i ‘root’ /etc/passwd

-c: Count matching lines

-c suppresses normal output and instead prints a count of how many lines matched. When multiple files are given, each filename is listed with its count.

grep -c ‘error’ /var/adm/messages

grep -c ‘FAILED’ /var/log/auth.log /var/log/syslog

This is often more useful than counting lines yourself with wc -l, because it counts lines, not occurrences — a line with two matches is still counted once.

-l: List filenames only

-l suppresses normal output and prints only the names of files that contain at least one match, one per line. This is ideal for finding which files in a tree contain a particular string without being swamped by the matching lines themselves.

grep -rl ‘PermitRootLogin’ /etc # find all config files mentioning this

grep -l ‘TODO’ *.c # find source files with TODO comments

Note: -l stops searching a file as soon as the first match is found, so it is faster than a normal grep over large files.

-n: Show line numbers

-n prefixes each matching line with its line number within the file. Useful when you need to go back and edit the file at the right location.

grep -n ‘MaxInstances’ /etc/ftpd.conf

-r + -R: Recursive search

-r searches all files under a directory, descending into subdirectories. -R does the same but also follows symbolic links to directories.

grep -r ‘connection refused’ /var/log

grep -rl ‘passwd’ /etc

-w: Whole word match

-w restricts matches to complete words. The pattern must be surrounded by non-word characters (spaces, punctuation, start/end of line). This prevents log matching login or syslog.

grep -w ‘log’ /var/adm/messages

-s: Suppress error messages

-s silences the errors grep would normally produce for files that don’t exist or can’t be read. This is useful in scripts where missing files are expected.

-h: Suppress filenames

When multiple files are given, grep normally prefixes each line with the filename. -h suppresses that prefix, giving clean output suitable for piping.

grep -h ‘error’ /var/log/*.log | sort | uniq -c

Context Lines: -A, -B, and -C

The -A, -B, and -C flags are found in BSD, GNU and other systems but not AIX. They are documented here because they are widely used systems and in cross-platform scripts. If you need this behaviour on AIX, the workaround is shown below.

On GNU/Linux systems, three flags control how many surrounding lines are shown alongside each match:

These are invaluable when reading log files, because a single matching line rarely tells the whole story — you need to see what led up to an error and what followed it.

# Linux only — not available on AIX grep

grep -A 3 ‘FAILED LOGIN’ /var/log/auth.log

grep -B 2 -A 5 ‘panic’ /var/log/syslog

grep -C 4 ‘out of memory’ /var/adm/messages

AIX workaround

On AIX, the -p flag prints the whole paragraph (text block separated by blank lines) that contains the match, which sometimes gives enough context. For precise line-count context, pipe through awk instead:

# Print 3 lines after each match using awk

grep -n ‘panic’ /var/adm/messages | awk -F: ‘

{ n=$1; for(i=n; i<=n+3; i++) print i }’ | xargs -I{} sed -n ‘{}p’ /var/adm/messages

Regular Expression Matching

By default, grep treats its pattern as a Basic Regular Expression (BRE). You do not need to use regular expressions — a plain word or phrase works fine. But knowing the basics lets you write much more powerful searches.

Anchors

Anchors match a position in the line rather than a character.

grep ‘^root’ /etc/passwd # lines starting with root

grep ‘sh$’ /etc/passwd # lines ending with sh

grep ‘^$’ report.txt # blank lines

The Dot: any character

. (dot) matches any single character except a newline.

grep ‘f.o’ file # matches foo, fao, f1o, f o, etc.

Character Classes

[…] matches any one of the characters listed inside the brackets.

grep ‘^[a-zA-Z]’ pgm.s # lines beginning with a letter

grep ‘[0-9]’ report.txt # lines containing any digit

Repetition

grep ‘err*or’ file # matches eror, error, errror …

grep ‘colou\?r’ file # matches color or colour

Escaping Special Characters

To search for a literal character that grep would otherwise treat as special (such as . or $), precede it with a backslash. Because the shell also interprets backslashes, you may need to double them or use single quotes:

grep ‘\$HOME’ script.sh # literal dollar sign

grep ‘192\.168’ hosts # literal dots in an IP address

Filtering Command Output

grep is very useful for filtering the output of a another command to find the lines you want.

Checking running processes

ps -ef | grep httpd # is the web server running?

ps -ef | grep -v grep # exclude the grep process itself

ps -ef | grep -i java # case-insensitive process search

The grep -v grep trick is necessary because ps -ef | grep httpd will always match its own grep process as well as the real target.

Checking network connections

netstat -an | grep LISTEN # show all listening ports

netstat -an | grep ‘:80 ‘ # is anything on port 80?

netstat -an | grep ESTABLISHED # active connections

Filtering filesystem and disk output

df -g | grep -v ‘Filesystem’ # disk usage, skip header line

lsdev -Cc disk | grep -i hdisk # list disk devices

mount | grep ‘/data’ # is /data mounted?

Inspecting installed filesets

lslpp -l | grep -i openssh # is openssh installed?

lslpp -l | grep COMMITTED # all committed filesets

User and group queries

grep ‘^frank’ /etc/passwd # frank’s passwd entry

grep -i ‘frank’ /etc/group # groups frank belongs to

who | grep frank # is frank logged in?

Searching Log Files

Log files on AIX are typically found under /var/adm/ and /var/log/ . At least on AIX systems that don’t store them in the ODM. The principles below apply equally to application logs wherever they are located, or indeed if you dump them from the ODM using the errpt command.

Finding errors and warnings

grep -i ‘error’ /var/adm/messages

grep -i ‘warn\|error\|fail’ /var/adm/messages

grep -c ‘error’ /var/adm/messages # how many error lines today?

Searching for a specific date or time

AIX syslog entries begin with a timestamp. You can anchor a search on the month and day:

grep ‘^Jun 8’ /var/adm/messages # all entries for 8 June

grep ‘^Jun 8 14:’ /var/adm/messages # entries for 14:xx on 8 June

Tracking a specific process or daemon

grep ‘sshd’ /var/adm/messages

grep ‘cron\[‘ /var/adm/messages # cron log entries (bracket is literal)

grep -i ‘lvmmon’ /var/adm/messages # LVM monitor messages

Finding login failures

grep -i ‘failed\|invalid\|illegal’ /var/adm/messages

grep ‘authentication failure’ /var/log/auth.log

Combining grep with other tools

Pipelines let you refine searches progressively or reformat the output:

# count errors per hour

grep ‘error’ /var/adm/messages | awk ‘{print $3}’ | cut -c1-2 | sort | uniq -c

# unique error messages, sorted by frequency

grep -i ‘error’ /var/adm/messages | sort | uniq -c | sort -rn | head -20

# errors from the last 100 lines of a log

tail -100 /var/adm/messages | grep -i error

# watch a log live and highlight errors

tail -f /var/adm/messages | grep –line-buffered -i error

Exit Status

grep’s exit status is useful in scripts for testing whether something exists:

# Test whether a user account exists

if grep -q ‘^frank:’ /etc/passwd; then

echo ‘Account exists’

fi

# -q suppresses all output — use purely for the exit code

grep -qs ‘error’ /var/adm/messages && echo ‘Errors found’

Quick Reference

grep [flags] pattern [file …]

Key flags:

-v Invert: print non-matching lines

-i Case-insensitive matching

-c Print count of matching lines

-l Print filenames only (not matching lines)

-n Prefix lines with line number

-r / -R Recursive (R also follows symlinks)

-w Whole-word match

-h Suppress filename prefix

-s Suppress file-not-found errors

-q Quiet: no output, use exit code only

-p Print whole paragraph containing match

-A n [BSD/GNU Only] Print n lines After match

-B n [BSD/GNU] Print n lines Before match

-C n [BSD/GNU] Print n lines Context around match

Pattern anchors:

^ Start of line

$ End of line

. Any single character

[abc] Any of a, b, or c

[^abc] Any character except a, b, or c

[a-z] Range of characters

* Zero or more of preceding

\+ One or more of preceding (BRE)

\? Zero or one of preceding (BRE)

\ Escape next character

What find Does

The find command walks a directory tree and, for each file it encounters, evaluates a series of expressions you supply. Each expression returns either true or false for that file. By default, find prints the path of every file for which the overall result is true.

An expression takes the form -something, usually followed by an argument. The expression is the test, the argument is what you’re testing for.

When you write several expressions one after another, find ANDs them together. A file must satisfy all of them to be selected.

find /data -type f -name “*.log”

This finds only plain files (-type f is true) and whose name ends in .log (-name “*.log” is true). If either test fails, the file is skipped.

Note that there are quotes around *.log. If there weren’t the shell would expand the *.log into a list of files, which would confuse the find utility, which expecting exactly one file specification. You could also quote the * with a backslash (\*) if you want to save a keystroke.

Combining Expressions

AND (default)

Placing two expressions in a row implies AND. Both must be true. You can actually have as many as you want and they’ll all be ANDed together. Although AND is implied, you can also specify it with -a if you think it improves readability. Newer versions of find support -and too.

find /home -user frank -size +10M

Finds files owned by frank and larger than 10 MB.

OR — -o

Use -o between expressions when either condition should match.

find /var -name “*.log” -o -name “*.tmp”

Finds files ending in .log or .tmp.

When mixing AND and OR, AND binds more tightly, just like multiplication before addition in arithmetic. Newer versions of find allow -or as well as -o.

Use parentheses (escaped from the shell) to make grouping explicit:

find /var \( -name “*.log” -o -name “*.tmp” \) -mtime +7

This finds .log or .tmp files that are also older than 7 days.

NOT — !

Place ! (or -not) before an expression to negate it. -not is sometimes more readable.

find /data -type f ! -name “*.bak”

Finds plain files whose name does not end in .bak.

Choosing the Starting Point

The first argument to find is always the directory to search from. find descends into sub-directories automatically. You can actually specify as many starting points as you like. They don’t look like expressions because they don’t begin with a -.

find / # searches everything (use with caution)

find . # searches from the current directory

find /var /home # searches both /var and -home

Common Expressions

-name and -iname

-name matches the filename (not the full path) against a shell-style pattern. The pattern must be quoted to stop the shell expanding it.

find /etc -name “*.conf”

-iname matches the filename but is case insensitive.

find /home -iname “readme*”

This matches README.txt, Readme.md, readme, and so on.

-type

Filters by file type. The most useful values on AIX are:

For example:

find /dev -type b

Find all block devices in /dev

Beware of not specifying a type, as you can get unexpected results with exec if you pass a directory to a utility when you thought it was getting just one file.

-mtime and -ctime

Both take a number of days.

-mtime n — the file’s modification time (data last written) is exactly n days ago.

-ctime n — the file’s change time (inode last changed — includes permission or ownership changes) is exactly n days ago. In other words, this is changes to the file’s metadata, not changes to its contents.

When using mtime or ctime you almost always use a + or – prefix:

find /logs -mtime +30 # not changed for over 30 days

find /tmp -ctime -1 # inode changed within the last day

Some versions of Unix, such as BSD, allow units other than days, as a suffix, such as s for seconds and h for hours. AIX may do so in the future, but not as of version 7.3.

-size

Matches on file size in blocks, and a block is 512 bytes.. As with the time expressions, other Unix systems allow a suffix, such as k for kilobytes, g for gigabytes, but as of AIX 7.3 this is not supported.

Again, + and – prefixes mean greater-than and less-than.

find /data -size +100 # files larger than 50K

find /tmp -size -2 # files smaller than 1K

-user and -group

Matches files owned by a specific user or group. You can use either the name or the numeric UID/GID.

find /home -user frank

find /projects -group dba

-perm

Matches on permission bits. The mode can be octal or symbolic.

find /bin -perm 4000 # files with the setuid bit set

find /tmp -perm 777 # files with exactly rwxrwxrwx

find /etc -perm 022 # files writable by group or other

Check the man page on the specific version of AIX you are using for symbolic representation as it has changed over time. It’s safer to stick with octal.

Finding Files with No Valid Owner: -nouser and -nogroup

find / -nouser

find / -nogroup

-nouser is true for any file whose numeric UID has no matching entry in /etc/passwd. -nogroup does the same against /etc/group.

These are useful after accounts are deleted — the files are left behind but owned by a UID that no longer maps to a name. On an AIX system using LDAP or NIS, the lookup is still done against the locally visible name database, so a file owned by a valid LDAP user will not be flagged unless that user is absent from the local resolution path.

What to Do With the Files Found

By default find simply prints each matching path. Several expressions change this behaviour.

-print

The default action — prints the full path, one per line. You rarely need to write it explicitly.

find /home -name “core” -print

-ls

Produces output in the format of ls -dils for each matched file: inode number, size in blocks, permissions, link count, owner, group, size in bytes, modification time, and name. Much more informative than -print for diagnostic work.

find /var/log -size +50M -ls

-exec

Runs an arbitrary command for each matched file. The string {} is replaced by the file’s path (this is the only time I’m aware of {} being used as a placeholder in Unix). The command must be terminated by \;. In fact, find is looking to end -exec with a semicolon, but this character has a special meaning to the shell so it’s necessary to use a \ or quotes to allow find to see it.

find /tmp -mtime +7 -exec rm {} \;

Deletes every file in /tmp not touched for more than 7 days.

find /home -name “*.log” -exec gzip {} \;

Compresses each .log file found using gzip.

The {} placeholder can appear more than once in the command if needed.

Running a command on all results at once with +

Replacing \; with + causes find to batch the paths and pass as many as possible to a single invocation of the command — much more efficient for commands like rm or chmod:

find /tmp -mtime +7 -exec rm {} +

Practical Examples

List all files larger than 100 MB modified within the last 30 days

find / -type f -size +200000 -mtime -30 -ls

This breaks down as:

• -type f — regular files only (skip directories, devices, and links)
• -size + 200000 — larger than 100 MB (e.g. 200,000 512-byte blocks)
• -mtime -30 — data modified within the last 30 days
• -ls — show full details rather than just the path

Move those files to /problems

find / -type f -size +100M -mtime -30 -exec mv {} /problems/ \;

Find and remove core dumps older than 14 days

find / -name “core” -type f -mtime +14 -exec rm {} \;

Find setuid files not owned by root

find / -type f -perm -4000 ! -user root -ls

Find files owned by deleted accounts

find /home -nouser -ls

Summary of Expression Syntax

find <path> [expression]

Expression operators:

expr1 expr2 AND (both must be true)

expr1 -o expr2 OR (either must be true)

! expr NOT (negates the result)

\( expr \) Grouping to override precedence

Common tests:

-name pattern Filename matches shell pattern (case-sensitive)

-iname pattern Filename matches shell pattern (case-insensitive)

-type [fdlbcp] File type

-mtime [+/-]n Modification time in days

-ctime [+/-]n Inode change time in days

-size [+/-]n[cMG] File size

-user name|uid Owned by user

-group name|gid Owned by group

-perm [-/]mode Permission bits

-nouser UID not in /etc/passwd

-nogroup GID not in /etc/group

Common actions:

-print Print path (default)

-ls Print detailed listing

-exec cmd {} \; Run command once per file

-exec cmd {} + Run command with batched file list

Microsoft has just open-sourced its Windows Subsystem for Linux (WSL).

https://blogs.windows.com/windowsdeveloper/2025/05/19/the-windows-subsystem-for-linux-is-now-open-source/

This is major. WSL runs the FOSS Unix knock-off on their closed source and expensive operating system, making it possible to host Unix applications on it. Cynics might think this was a ploy to still sell a Windows server license instead of people running Linux direct on the hardware. Or you could say it allows lower skilled Windows administrators who couldn’t cope with a command line to still access Linux applications.

Since it first appeared, people have been questioning Microsoft’s open source credentials, as WSL was closed source. Not now. You can get at the source code, customise it and run your own version.

This is great news, but as with anything Microsoft, it’s probably another cyber security attack vector for Windows.

I’ve seen all sorts of stuff on forums about how to process command line argument in C or C++. What a load of fuss and bother. There’s a standard getopt() function in the ‘C’ library, similar to the shell programming command, but it’s not great.

The main problem with getopt() is that it produces its own error message. Although this saves you the trouble, it can be a bit user unfriendly for the user, especially when things get complex. For example, you might have mutually exclusive arguments and want to print out a suitable message. That said, it’ll work most of the time.

Here’s a page showing a good explanation for the GCC version, whcih is pretty standard.

Example of Getopt (The GNU C Library)

But rolling your own is not hard. Here’s a skeleton I use. It’s pretty self-explanatory. My rules allow single options (e.g. -a -b) or combined options (e.g. -ab), or any mixture. “–” ends options, meaning subsequent arguments are going to be a actual arguments.

If you want to pass something as an option, such as a filename, you can. -fmyfile or -f myfile are both handled in the example.

You can add code to detect a long option by adding “if (!strcmp(p,”longname”)) … just after the char c. But I don’t like long options.

#include <stdio.h>

void process(char *s)
{
    printf("Processing %s\n",s);
}

int main (int cnt, char **a)
{
    int i;
    for (i=1; i<cnt && a[i][0] == '-'; i++)
    {
        char *p = &a[i][1];
        char c;
        if (*p == '-')
        {
            i++;
            break;
        }
        while (c = *p)
            switch (*p++)
            {
            case 'a':
                printf("Option a\n");
                break;
                
            case 'b':
                printf("Option b\n");
                break;

            case 'f':
                if (!*p && i+1 < cnt)
                    printf("Value for f=%s\n", a[++i]);
                else
                {
                    printf("Value for f=%s\n", p);
                    while (*p)
                        p++;
                }
                break;
                
            default:
                printf("Bad switch %c\n",c);

            }
    }
    for (;i<cnt;i++)
        process(a[i]);
}

The above code assumes that options precede arguments. If you want to mix them the following code allows for complete anarchy – but you can end it using the “–” option, which will take any following flags as arguments. As a bonus it shows how to add a long argument.

#include <stdio.h>
#include <string.h>

void process(char *s)
{
    printf("Processing %s\n",s);
}

int main (int cnt, char **a)
{
    int i;
    int moreargs=1;

    for (i=1; i<cnt; i++)
    {
            if (moreargs && a[i][0] == '-')
            {
            char *p = &a[i][1];
            char c;
                if (*p == '-')
                {
                    moreargs = 0;
                    continue;
                }
                if (!strcmp(p,"long"))
                {
                    printf("Long argument\n");
                    i++;
                    continue;
                }

                while (c = *p)
                    switch (*p++)
                    {
                    case 'a':
                        printf("Option a\n");
                        break;

                    case 'b':
                        printf("Option b\n");
                        break;

                    case 'f':
                        if (!*p && i+1 < cnt)
                            printf("Value for f=%s\n", a[++i]);
                        else
                        {
                            printf("Value for f=%s\n", p);
                            while (*p)
                                p++;
                        }
                        break;

                        default:
                            printf("Bad switch %c\n",c);

            }
        }
    else
        process(a[i]);
    }
}