Using Parameters and Variables

You saw in chapter 5 that, when Qshell finds a dollar sign preceding a parameter or variable name , it replaces the reference with the value of the indicated parameter or variable. This is known as expanding the parameter or variable.

Both the terms parameter expansion and variable expansion are used in Unix literature to refer to this process, no matter whether parameters or variables are being discussed. This chapter uses the term variable expansion because all facets of expansion are supported with variables, but one type of expansion is not supported with positional parameters.

Variable Expansion

In its simplest form, variable expansion returns the value assigned to a parameter or variable. For example, $1 and ${1} expand to the value of the first positional parameter, and $dirname and ${dirname} expand to the value of a variable whose name is dirname . However, Qshell provides operators that permit more powerful methods of variable expansion.

Table 6.1 lists the four expansion operators. Use these operators within the brace form of a variable or parameter. Before the closing brace , include the operator, and follow it with another value, whose function depends on the operator.

The term missing value in Table 6.1 needs some explanation. You might remember from chapter 5 that there is a difference between a variable that is unset and a variable that has a null value. An unset variable is not defined, while a variable with a null value is defined, but empty.

If the operators in Table 6.1 are preceded by a colon , the term missing value means "an unset variable or a variable with the null value." Without a preceding colon, the term missing value means "an unset variable." As a rule, you will probably want to include the colon, since your consideration when using these parameter operators is more likely to be whether or not a value is missing, not why it is missing. Table 6.2 illustrates this difference more closely. Keep the following in mind to help interpret the table:

• $ var indicates that Qshell returns the existing value of the variable.
• Value indicates that Qshell returns the value following the operator.
• Assign value to var means that the value following the equal sign is assigned to the variable.
• Null indicates that Qshell uses a null value in place of the variable reference.
• Error means that Qshell displays the message following the question mark, or a default message, and exits a noninteractive shell.

The next several sections of this chapter examine each of the variable expansion operators in detail.

Temporary Substitution for a Missing Value

Use the hyphen operator to temporarily substitute a value when a variable has no value. For example, in Figure 6.1, the variable name is undefined (unset), so the first print command produces a blank line. The second print command prints the value Joe Smith because name has no value. However, the second print does not assign a value to the name variable, as the third print proves. Once the value Jack Sprat is assigned to name , the value following the hyphen is not used.

print $name /home/JSMITH $ print ${name:-Joe Smith} Joe Smith /home/JSMITH $ print $name /home/JSMITH $ name='Jack Sprat' /home/JSMITH $ print ${name:-Joe Smith} Jack Sprat

Figure 6.1: The presence of a colon causes null and unset values to be treated equally.

Figure 6.2 provides another example of the hyphen operator. In this case, the first print statement prints the value Bob because the variable name is undefined (unset). The second print does not print Bob because no colon precedes the hyphen operator, and name is null, but not unset.

print ${name-Bob} Bob /home/JSMITH $ name= /home/JSMITH $ print ${name-Bob} /home/JSMITH $

Figure 6.2: This example illustrates the difference between using the hyphen operator with an unset variable and a null variable.

Temporary Substitution of an Existing Value

Qshell also provides a way to temporarily override the existing value of a variable. If the variable has a non-null value, the plus operator substitutes an alternate value. Otherwise, the result of the expansion is a null value.

Consider Figure 6.3. The first two print commands print blank lines because name is undefined . The third one prints Bubba even though name has the value Bill . The last print shows that the plus operator did not change the value of variable name .

print $name /home/JSMITH $ print ${name+Bubba} /home/JSMITH $ name=Bill /home/JSMITH $ print ${name+Bubba} Bubba /home/JSMITH $ print $name Bill

Figure 6.3: These commands illustrate the plus operator.

Assigning a New Value for a Missing Value

Use the equal-sign operator to assign a new value to a variable with no value. That is, the variable continues to have the new value after the expansion. This type of expansion is not allowed for positional parameters.

Figure 6.4 illustrates the equal-sign operator. The variable name is unset, so the first print command leaves a blank line. The variable expansion in the second print assigns the value Suzy Q to name . The third print shows that name has kept its new value.

print $name /home/JSMITH $ print ${name:=Suzy Q} Suzy Q /home/JSMITH $ print $name Suzy Q

Figure 6.4: The equal-sign operator assigns a new value to a variable with no value.

Generating an Error If a Value Is Missing

The question-mark operator causes Qshell to issue an error message if a variable has no value. For example, the first three print commands in Figure 6.5 send messages to stderr because name is unset. The last print command does not send an error message because name has a value.

print ${name?} qsh: 001-0021 Parameter 1$.*s is not set. /home/JSMITH $ print ${name:?} qsh: 001-0022 Parameter 1$.*s is not set or is null. /home/JSMITH $ print ${name:?Name is undefined } Name is undefined /home/JSMITH $ name='Larry, Curly and Moe' /home/JSMITH $ print ${name:?Name is undefined } Larry, Curly and Moe

Figure 6.5: These commands illustrate the question-mark operator.

Pattern Modifiers

You can use pattern modifiers to do certain types of string manipulation. Among other uses, pattern modifiers are most frequently used in scripts to manipulate variables containing file name or path name values. The four pattern modifiers are listed in Table 6.3.

These operators do not modify variables. Rather, each one returns a new value you can use in Qshell commands. You can assign the value to another variable, print the value, etc.

Figure 6.6 shows each of the pattern modifiers at work on the variable var , which contains a list of consecutive letters interspersed with x 's:

• The # operator, working from the beginning of the string, removes the smallest pattern that consists of zero or more characters followed by an x , which is abcdex .
• The ## operator, working from the beginning of the string, removes the longest pattern that consists of zero or more characters followed by an x , which is abcdexfghxijklmnxopx .
• The % operator, working from the end of the string, removes the smallest pattern that consists of an x followed by zero or more characters, which is xqrst .
• The %% operator, working from the end of the string, removes the longest pattern that consists of an x followed by zero or more characters, which is xfghxijklmnxopxqrst .

print $var abcdexfghxijklmnxopxqrst /home/JSMITH $ print ${var#*x} fghxijklmnxopxqrst /home/JSMITH $ print ${var##*x} qrst /home/JSMITH $ print ${var%x*} abcdexfghxijklmnxop /home/JSMITH $ print ${var%%x*} abcde

Figure 6.6: Each of the pattern modifiers produces a different result from the same string.

Figure 6.7 shows additional ways to use the pattern modifiers to manipulate strings. Here, the % is used to extract the directory name because it removes as few characters as possible from the end of the string. The ## operator is used to extract the file's base name because it trims all characters as possible through the last slash.

filename=/home/jsmith/temp/work1.txt /home/JSMITH $ dir=${filename%/*} /home/JSMITH $ print $dir /home/jsmith/temp /home/JSMITH $ file=${filename##*/} /home/JSMITH $ print $file work1.txt /home/JSMITH $

Figure 6.7: You can use the pattern modifiers with many different Qshell commands.

Substrings

With V5R2, Qshell includes a form of variable expansion that allows you to retrieve substrings. It is similar in syntax to substring constructs found in RPG, CL, COBOL, and Java. Here is the substring syntax:

${variable:offset [:length] }

The offset parameter indicates the position at which to begin extracting the substring. The value indicates how far that character is offset from the beginning of the string; thus, the first character is at offset zero, not one. Another way to think of the offset is "the number of characters to skip." That is, if you skip zero characters, you begin extraction at the first byte. If the offset is longer than the string, Qshell returns the empty string. The length parameter is optional. If no length is specified, Qshell returns all characters from the offset through the end of the string.

Figure 6.8 demonstrate the use of the substring construction. In Figure 6.8, the first substring operation skips five characters, to begin extracting the remainder of the string at the sixth character. The second substring operation extracts two characters only. The remaining substring operations in Figure 6.8 show that you may use arithmetic expressions as offset and length arguments.

somevar=abcdefghij /home/smith $ print ${somevar:5} fghij /home/smith $ print ${somevar:5:2 } fg integer i=4 len=3 /home/smith $ print ${somevar:i:len} efg /home/smith $ print ${somevar:i+1:len} fgh print ${somevar:i+1:len+1} fghi

Figure 6.8: The first two print commands show the basic use of the substring syntax. The remaining commands use more complex syntax, involving mathematical expressions.

Substitution Expressions

V5R2 also introduced two new substitution expansion structures:

${variable/pattern/string} ${variable//pattern/string}

Notice that there is no trailing slash after the replacement string.

Figure 6.9 provides a simple example of a substitution expression.

print $name Joe Smith /home/JSMITH $ print ${name/Joe/Fred} Fred Smith

Figure 6.9: The second expression substitutes Fred for Joe .

As Figure 6.10 shows, if the first slash is doubled, all instances of the longest match of the pattern are replaced . If the first slash is not doubled , only the first instance of the longest match of the pattern is replaced.

print $title Director of the Department of Redundancy Department /home/JSMITH $ print ${title/Department/Bureau} Director of the Bureau of Redundancy Department /home/JSMITH $ print ${title//Department/Bureau} Director of the Bureau of Redundancy Bureau

Figure 6.10: In the first substitution, only the first instance of Department is changed to Bureau . In the second, both instances are changed.

A leading % in the pattern means that the pattern must match at the end of the variable, as shown in Figure 6.11. If no replacement string is given, the pattern is removed, and a leading # in the pattern means that the pattern must match at the beginning of the variable. Figure 6.12 illustrates these points.

print $dept The Department of Redundancy Department /home/JSMITH $ print ${dept/Department/Bureau} The Bureau of Redundancy Department /home/JSMITH $ print ${dept/%Department/Bureau} The Department of Redundancy Bureau print ${title/#Director/Supervisor} Supervisor of the Director of the Department of Redundancy Department /home/JSMITH $

Figure 6.11: The percent sign in the third expansion means that the pattern must match at the end of the variable. Therefore, the second occurrence of Department is changed to Bureau , not the first.

print $title The Director of the Director of the Department of Redundancy Department /home/JSMITH $ print ${title/Director of the /} The Director of the Department of Redundancy Department title="Director of the Director of the Department of Redundancy Department" /home/JSMITH $

Figure 6.12: Since no replacement string is given in the first expansion, the first instance of Director of the is removed. The second expansion changes Director to Supervisor because Director is found at the beginning of the variable.

Finally, the new substitution expansion structures can be used to strip special characters , as shown in Figure 6.13. The variable music contains leading and trailing double quotes. The expansion removes them. Notice that the double-quotes character is prefixed with a backslash in the expansion because of its special meaning to Qshell

print $music "Le Sacre du Printemps" /home/JSMITH $ print ${music// "/} Le Sacre du Printemps

Figure 6.13: Substitution expansion structures can be used to remove special characters from a string.

Finding the Length of a Value

Qshell includes an expansion operator that returns the length of the value stored in a variable. Here is the syntax:

${#variable}

Figures 6.14 uses the length expansion operator in a simple expression.

print $filename /home/jsmith/temp/bin/mydata.txt /home/JSMITH $ print ${#filename} 32

Figure 6.14: The # preceding the variable name indicates that the length of the value of the variable is to be retrieved.

Here is the operator used in a more complex expression:

if [[ ${#1} -lt 5 ]] then print "First parameter must be at least five characters long." >&2 exit 2 fi

In this example, if the length of the first positional parameter is less than five characters, Qshell sends an error message to the standard error device and exits the script, with an exit status of two.

Concatenating Strings

While many other languages have concatenation operators, Qshell does not. Qshell does not need a concatenation operator because concatenation is achieved by abutting values. For example, in Figure 6.15, the filename variable is assigned the value of the path variable followed by a slash and the value of the file variable.

print $path /home/JSMITH /home/JSMITH $ print $file cust.csv filename=$path/$file /home/JSMITH $ print $filename /home/JSMITH/cust.csv cat $filename 1,10001,"Pye Cherry",25 1,20002,"Moss Pete",25.5 1,30003,"Dover Ben",35 1,40004,"Bellum Sara",45 2,1,"Gunn Tommy",20 2,3,"Unsaturated Polly",35 2,345,"Sox Bobby",-9.75

Figure 6.15: Concatenation is achieved by abutting values.

To include blanks in a concatenation, enclose the entire assigned string in double quotes, as shown in Figure 6.16. Single quotes would prohibit the variable expansion.

firstname=Willie /home/JSMITH $ lastname=Makit /home/JSMITH $ fullname="$firstname $lastname" /home/JSMITH $ print $fullname Willie Makit

Figure 6.16: To include blanks in a concatenation operation, enclose them in double quotes.

Numeric Constants

Qshell supports two types of numeric values: integer and floating-point (real). Integer values are of the form base#value , where base is a number between two and 36. If base# is omitted, the value is assumed to be base-10. The letters of the alphabet are used as digits in bases greater than 10, where the letter A represents 10 and the letter Z represents 35. The letters may be in either uppercase or lowercase.

Figure 6.17 contains references to numbers in various bases. The first printf command prints the values of a , b , and c in decimal format. The second printf prints the values of those variables in hexadecimal format.

a=59 b=2#1101 c=16#2F /home/JSMITH $ printf "%d %d %d " $a $b $c 59 13 47 /home/JSMITH $ printf "%X %X %X " $a $b $c 3B D 2F

Figure 6.17: Integer values may be specified in any base from two to 36. The default base is 10.

Floating-point values may contain fractional portions. They are of the following form:

[+-]number[.number][exponent]

The exponent is the letter E (or e ) followed by a number. Figure 6.18 shows examples of floating-point numbers. The first assignment statement assigns values, specified in three different formats, to variables x , y , and z . The second assignment statement assigns values in yet more formats to variables a through f . The two printf commands display the assigned values as floating-point numbers.

x=2 y=4.5 z=1.7e10 /home/JSMITH $ printf "%f %f %f " $x $y $z 2.000000 4.500000 17000000000.000000 /home/JSMITH $ a=39 b=-55 c=14.246 d=-294.8832 e=1.7e10 f=-2.55e4 g=3.93e-2 h=-7.4599e-2 /home/JSMITH $ printf "%f " $a $b $c $d $e $f 39.000000 -55.000000 14.246000 -294.883200 17000000000.000000 -25500.000000

Figure 6.18: Floating-point numbers may be specified in several ways.

By default, arithmetic is done with integers. To enable floating-point arithmetic, use the float option of the set command. This option may be specified in two ways:

set -F set -o float

In Figure 6.19, two variables, x and y , are multiplied, and the product is stored in z . When integer arithmetic is active, the multiplication operation fails, and z is unchanged. When floating-point arithmetic is active, the multiplication succeeds.

print $x $y $z 2 4.5 4 /home/JSMITH $ let z=x*y let: 001-0032 Number 1.7e10 is not valid. /home/JSMITH $ print $z 4 /home/JSMITH $ set -F /home/JSMITH $ let z=x*y /home/JSMITH $ print $z 9 /home/JSMITH $

Figure 6.19: Use the set utility to enable floating-point arithmetic.

Arithmetic Expressions

An arithmetic expression is a string of tokens that is interpreted as a numeric calculation. The tokens may include variable and parameter names, numeric literals, operators, and parentheses. Variable names used in arithmetic expressions do not have to be preceded by dollar signs, but it is not incorrect to include dollar signs. An unset or null variable is interpreted as zero when used in arithmetic expressions.

The supported arithmetic operators are listed in Table 6.4. They are shown in the table in order of precedence, but you may use parentheses to override their default order.

You can use arithmetic expressions in the following locations:

• In an arithmetic expansion
• In an argument of the let utility
• In the argument of the shift utility
• In the arithmetic formats of the printf utility
• In arguments of the test utility
• In the argument of the ulimit utility
• In the offset and length parameters of the substring variable expansion

The following example demonstrate the use of an arithmetic expression in an argument to the shift utility:

shift nb+2

The shift utility adds two to the value in variable nb and causes Qshell to shift the positional parameter list by that number of parameters. You could also have added the dollar sign, as follows (although it is not necessary):

shift $nb+2

The Let Utility

The let utility is the usual way to perform arithmetic, especially when you want to assign the result of a calculation to a variable. Here is the syntax of let :

let arithmetic- expression ...

You may place more than one expression on a line. The expressions are evaluated from left to right.

Figure 6.20 provides an example so you can better understand the let utility. In it, the value assigned to c is based on the result of the calculations for a and b . Before calculating a value for c , Qshell assigns values 12 and 60 to a and b respectively. For that reason, c receives a value of 5, not 2.

print $a $b $c 10 20 30 /home/JSMITH $ let a=a+2 b=b*3 c=b/a /home/JSMITH $ print $a $b $c 12 60 5

Figure 6.20: The let utility evaluates the expressions from left to right.

You can use the let utility in several different ways. Here are a few examples, which all add one to the value of variable j :

let j+=1 let j=j+1 let "j=j+1"

Arithmetic Expansion

Arithmetic expansion allows you to embed the result of an arithmetic expression within a command. The syntax is as follows:

$(( expression ))

Simple arithmetic expansion is demonstrated in the following arithmetic expansion, which examines the remainder after first dividing variable year by four, and subsequently dividing by 100 and 400 to determine whether or not year is leap year:

if [ $((year % 4)) -eq 0 ] then if [ $((year % 100)) -ne 0 -o $((year % 400)) -eq 0 ] then let count-=1 fi fi

In Figure 6.21, the values in positional parameters 1 and 5 are added together within the printf command.

print 10 50 /home/JSMITH $ printf "%d " $(( + )) 60

Figure 6.21: Arithmetic expansion is used to perform arithmetic within Qshell commands.

The positional parameters $1 and $5 in Figure 20 must be preceded by dollar signs in order to distinguish them from the literals 1 and 5. However, variables do not have to be preceded by dollar signs. In Figure 6.22, the arithmetic expansion yields the same result, whether or not the length variable is preceded by a dollar sign.

size=$((length+1)) /home/JSMITH $ print $size 26 /home/JSMITH $ size=$(($length+1)) /home/JSMITH $ print $size 26

Figure 6.22: Prefixing a dollar sign to a variable name usually has no effect in arithmetic expansion.

However, in the case of assignment commands, the presence or absence of a dollar sign does matter. In Figure 6.23, for example, the second print command assigns a value to variable name because no dollar sign precedes the variable.

a=10 b=20 c=30 /home/JSMITH $ print $a $b $c $name 10 20 30 /home/JSMITH $ print $((name=999)) 999 /home/JSMITH $ print $a $b $c $name 10 20 30 999 /home/JSMITH $

Figure 6.23: Dollar signs should usually be omitted when assignment statements are used in arithmetic expressions.

Compare Figure 6.23 to Figure 6.24, where variable name is preceded by a dollar sign in the echo command. The arithmetic expansion changes variable b , not variable name , because Qshell expands $name to b .

a=10 b=20 c=30 name=b /home/JSMITH $ print $a $b $c $name 10 20 30 b echo $(($name=999)) 999 /home/JSMITH $ print $a $b $c $name 10 999 30 b /home/JSMITH $

Figure 6.24: Prefixing the name variable with a dollar sign causes a different variable to be modified.

The Expr Utility

The only thing to remember about using the expr utility to carry out arithmetic is this: "Just say no!" Here are some reasons why:

• Qshell starts a new subshell to run expr . Starting a subshell degrades performance. Arithmetic expansion runs in the same shell.
• Many of the operators that expr uses have special meanings to Qshell, and therefore have to be escaped with the backslash character.
• The expr utility handles integer arithmetic only.
• Arguments must be separated from one another by white space.
• The expr utility is an old one, from the Bourne shell. Newer arithmetic mechanisms have since been invented.
• The expr utility writes the result of an expression to standard output, rather than assigning a value directly to a variable.

The expr utility was good for its time, but better ways of doing arithmetic in shells have been invented since then.

Summary

Variable expansion is the process of replacing the token that represents a parameter or variable with a value. Variable expansion allows you to temporarily assign a value to a variable, assign a value to a missing variable, manipulate strings, and extract substrings.

Arithmetic expansion provides a way for you to carry out arithmetic in Qshell scripts. Some Qshell utilities, such as let , shift , and test , also have limited arithmetic abilities .