Shell tutorial part 3

From LXF Wiki

(Difference between revisions)
Revision as of 20:28, 29 May 2008
Towy71 (Talk | contribs)

← Go to previous diff
Revision as of 22:26, 29 May 2008
Towy71 (Talk | contribs)

Go to next diff →
Line 1: Line 1:
- +(Original version written by Marco Fioretti for LXF issue 67.)
PART 3 In which teacher's pet Marco Fioretti adds weight to Nick's argument that the PART 3 In which teacher's pet Marco Fioretti adds weight to Nick's argument that the
console can do graphics with a lesson for creating an image gallery. console can do graphics with a lesson for creating an image gallery.

Revision as of 22:26, 29 May 2008

SHELL SECRETS COMMAND LINE SERIES (Original version written by Marco Fioretti for LXF issue 67.) PART 3 In which teacher's pet Marco Fioretti adds weight to Nick's argument that the console can do graphics with a lesson for creating an image gallery.

Table of contents

Shell Arrays, dialogs and images

Our divine mission to open up the command line continues. We start this month's tutorial with arrays of variables ­ making accessible something that was previously reserved for IT students. We will then move on to techniques that will take a bit of the frustration out of using the CLI: getting the script to ask for instructions and teaching the console a bit of maths. Finally, lest people think that a text interface can only handle text, we'll play a bit with perhaps the most mainstream home-computer activity imaginable: handling digital photos. As usual, a final script will provide a real-world application ­ a thumbnail gallery, no less ­ of the concepts explained.

Writing arrays

Many variables have what we call a scalar nature: in normal language, this simply indicates that they are one single piece of content, not fragmented or structured into separate parts. Things like $YOUR_HEIGHT or $LINUS_LAST_NAME belong to this category. Every programming language can handle this kind of variable, and most languages also support more complicated data structures and have customised operators to manage them efficiently. In comparison, the various Unix shells are pretty limited, because they were never intended for high-level data manipulation. However, bash, the reference Linux shell we've used so far in this series, is quite capable of handling basic arrays, which are unidimensional containers of scalars. Unidimensional means that an array can be thought of as one ordered stack of separate values ­ rather than as a table, where data is arranged in rows and columns. You can access any element in an array just by giving its numeric distance from the beginning of the array itself. So the first element of A_4_PIECES_ARRAY will be $A_4_PIECES_ARRAY[0] and the last $A_4_PIECES_ARRAY[3]. Shell arrays do not need to have all their elements explicitly defined ­the first line of your script can be something like this:

 FRIENDS[3] ='Peter'

This creates the FRIENDS array, with its first three elements (indexes 0, 1 and 2) empty.

    However, it's good practice (especially in complex scripts) to explicitly declare an array as follows:
declare -a FRIENDS

Arrays can be filled in more efficiently than by assigning values one at a time, thus:

FRIENDS=( Martin Karl `Jean Luc' Peter)
FRIENDS=( `cat somefile.txt | tr `\n' ` ``)

The first instruction just assigns the elements in the desired order. Note the single quotes keeping `Jean Luc' as a single value despite the space between Jean and Luc. The second instruction uses command substitution to fetch the whole content of a text file (cat), put it all on one line by changing new lines (\n) to white spaces with tr and use the result as a words list for the array.

When it's finally time to use an element, this is the right syntax:

linux->echo ${FRIENDS[2]}
Billy Jean

Don't forget the curly braces or Weird Things will happen:

linux->echo $FRIENDS[2]

Here, bash doesn't look at FRIENDS as an array any more ­ it just remembers its first part as a scalar, and adds to it the other characters we typed.

reading input

We already know that a script can be given different command-line parameters every time it's launched. These are internally accessible through the built-in variables $1, $2 and so on, but there's a major limitation to this approach: you have to give the script all the answers it might need before you launch it. What would really make an enquiry like this useful is if the script could do something, report and (depending on what happened) ask the user for further instructions in real time ­ without forcing us to start all over.

Don't worry. It is possible to ask for, and load, user input while a script is running. This is accomplished with the read command, which reads the value of one or more variables from STDIN (where you type from).

echo -n "Who is your best friend?"
     echo "Your Best Friend is $BEST_FRIEND"

If you don't give read the name of a target variable it will save what you entered into the built-in variable $REPLY. To read in more than one variable with only one call you would list all their names in a row:


Or, if you wanted all the values to go into an array, all nicely indexed, you would say so with the -a switch:


This would store all the names you enter in ${FRIENDS_LIST[0]}, ${FRIENDS_LIST[1]} and so on. Last but not least, read can get its input automatically, from an assigned file: we will see how to do this in the final example.

and arithmetic

Occasionally, shell scripts can also do some maths work. They won't perform as well as languages designed specifically for such tasks, but they're more than acceptable for everyday light use. There are three ways to perform arithmetic operations in the shell. The first one is to launch the expr command inside inverted quotes:

  Q=1;Q=`expr 4 + $Q \* 3`;echo $Q

Yes, this does look like one of LXF Towers' tame monkeys randomly hitting the keyboard. But the result, instead of a Shakespeare sonnet, is 7. Why? Well, first the Q variable is initialised. Then we ask expr to multiply it by 3 and add 4 to the result. The inverted quotes make the result, ie 7, go back into the Q variable. Note the back slash before the multiplication operator *: ­without it the shell would have seen a metacharacter meaning (in that context) `the names of all the files in the current directory'. Not what you would use for calculations, right? A more popular construct for calculations is with double parentheses:


This form lets you use spaces, making it all a bit more readable, and also supports a C-like format:

  ((a += 1))

Another admissible syntax is the let operator:

  let Q=Q+10
  let "Q += 10"

An art lesson to finish

The ImageMagick toolbox ( is a collection of little programs for manipulating images in many, many ways, either with a GUI or straight from the command line. ImageMagick is included in just about any desktop-oriented GNU/Linux distribution, but many users ignore its existence or only know its graphic front-end, the display program. The true value of this package, however, lies in its command-line pieces, because they can be used to perform repetitive operations on many images at the greatest possible speed. Here are some basic examples of ways in which you can use ImageMagick's most popular components (check the web page, there are many more):

  convert -geometry 200x200 some_big_picture.jpg small_
  convert -fill white -font helvetica -pointsize 100 -draw "text
100, 00 \"Wonderful flowers" original.jpg final.jpg

What's happening here? The first command is just rescaling some big images (Fig 1) to 200x200 pixels. The second one is adding the caption to the flowers picture: it specifies the caption text and position as well as the font family and size (Fig 2). Convert can also be used to create simple animations: check the website or the man page for details. If, instead of text, you wanted to add a graphic logo to each image, the ImageMagick tool of choice would be Combine. Used as in this example, it would add your logo in the bottom-left-hand corner:

  combine -gravity SouthWest -compose Over some_picture.png
  my_logo.gif picture_with_logo.png

Of course, there's nothing to stop you doing both of these things ­ that is, cascading the commands above so that you have both a logo and a caption in the final image. Keep in mind one thing, however: JPEG and other common image formats discard some data in the compression process; meaning that when you save an image as a JPEG file it loses some quality. Consequently, a JPEG-modified version of a JPEG modified-version of a JPEG original might not be worth much at the end of the day. Other, non-degrading formats should be used for repeated editing. For a very brief introduction to these issues, check out the page OK, let's get to work, putting everything we've learned in this tutorial together. Imagine that you have, every now and then, a pile of images in a folder. Say you want to put smaller versions of all the pictures online, each with its own overwritten comment. Start by writing a caption for each image and saving them in a separate text file, in this format:

picture_1.jpg Junior learning to swim
picture_2.jpg Daddy tasting the grilled lobster

Preparing the file above would be the only time-consuming part ­ the script using it would only be a few lines long.

while [ 1 ]
    read IMAGE_NAME CAPTION || break
    convert -sample 25%x25% -draw "text 10, 0 \"$CAPTION"
$IMAGE_NAME.jpg ${IMAGE_NAME}_thumbnail.jpg
    SIZE_OF_THIS_IMAGE=`ls -s ${IMAGE_NAME}_thumbnail.
 done < $1
echo "The total space needed on the website will be $TOTAL_

The while/do/done magic makes the script loop over the whole file provided as first argument ($1), one line at a time (we'll discuss shell control structures and loops in detail next month). At each iteration, the read instruction loads the first string into IMAGE_NAME. Everything else after the first space is dumped (because there are no other variables listed) into CAPTION, just as we'd like. The script generates a thumbnail image with the same name as the original but with a _thumbnail extension. The picture's height and width are reduced to one quarter of the original value (-sample 25%x25%). Next, the size in bytes of the resulting thumbnail is stored in SIZE_OF_THIS_IMAGE and added to TOTAL_SPACE_NEEDED. When there are no more lines to read, the loop is interrupted (note the break keyword) and we end this month's tutorial being told how many more megabytes of web storage we will have to buy... LXF