The power (but enormous setup cost) of scripting
May 2007 (perspective of a Ph.D. student)
Earlier today ...
My friend asked me whether I knew of any photo editing programs that he could use to easily resize a batch of digital photos. Since he had hundreds of photos, he didn't want to open a photo editing program such as Adobe Photoshop and manually resize and save all of them. The first thought that popped into my head was ImageMagick, a suite of free command-line photo manipulation tools. I knew that ImageMagick contained a program that could resize photos, and that I could just write a script to use that program to resize all the photos in his batch. I told my friend that if he could get ImageMagick installed on his Windows machine, I could quickly write him a short script that could do what he wanted.
Command-line programs versus GUI programs
Before I proceed with my story, I want to make sure that you, the reader, understand the distinction between a GUI program and a command-line program. The vast majority of computer users today interact solely with GUI programs on a daily basis, usually on a Windows-based operating system. GUI stands for Graphical User Interface, and a GUI program is one in which the user interacts with the program by using the mouse to click around on graphical elements on the screen and occasionally using the keyboard to type in some text. Pretty much every program you probably use is a GUI program, from your web browser to your email client to your word processor. Most people think that GUI programs are the only type of computer programs that exist, but there is another type: command-line programs.
A command-line program does not contain any graphical elements that a user can interact with; instead, a user runs a command-line program by typing the program's name along with its inputs into a text window called a terminal (or a 'command prompt') and then hitting the Enter key. The user must then wait until the program is finished, at which point he can enter in another program to launch. (There are exceptions to the non-interactive nature of command-line programs—some command-line programs such as text-based email clients are actually interactive, and you can think of them as GUI programs except that they execute within the confines of a terminal, but in this article, I will only consider non-interactive command-line programs.) At first glance, this non-interactive, command-based technique seems like an esoteric and inefficient way to use computers, but command-line programs have certain strengths over GUI programs (and vice versa, of course); most notably, command-line programs can be easily combined together via user-written scripts in order to make the computer perform repetitive tasks rather than making humans do them. I'll demonstrate with the following example:
convert big_bunny.jpg -resize 25% little_bunny.jpg
The first word I type,
So this is actually kind of cool, because in order to do the same thing with a GUI program like Adobe Photoshop, I would need to perform the following steps:
In total, it could take anywhere from 1 to 2 minutes to resize one photo, depending on how fast you are with the mouse. The reason it takes so long is that Photoshop can do much more than simply resizing photos, so it takes time to navigate around its menus to select just the photo resizing functionality. In contrast, it took approximately 10 seconds to type in the following command
convert big_bunny.jpg -resize 25% little_bunny.jpg
hit the Enter key, and wait half a second for the program to complete. The advantage of the command-line program over the GUI program in this application is that it is much faster to accomplish a single specific non-interactive task such as resizing an image. On the flip side, the GUI program is far better for interactive tasks such as making line drawings on an image with the mouse. (try doing THAT by just typing in textual commands in a terminal!)
I hope I've convinced you that, for at least this particular image opening, resizing, and saving task, using a command-line program saves you some time over using a GUI program (command-line takes approximately 10 seconds whereas GUI takes approximately 90 seconds). But if you only want to resize one image, who the heck cares? 90 seconds isn't unbearably long. And if you already have Photoshop open, it might take you only 60 seconds to resize each subsequent image. It might not be worth the hassle to learn how to use command-line programs and just stick with GUI programs like Photoshop for these tasks. After all, it's much more intuitive and less error-prone to select menu items using the mouse than having to remember to type in a sequence of commands in the terminal in EXACTLY the right order and manner. Hmmm, there must be some other advantage of command-line programs that make their lack of user friendliness worth it ...
The power of scripting
Now, back to my friend's problem: He didn't just have one image that he wanted to resize; he had HUNDREDS. How long would it take to manually open each file in Photoshop, click around the menus to find the resize dialog box, type in "25" for 25%, click "OK", and save the modified file under a new name? It's simple multiplication. Let's say that you're a real speed demon and can click around so fast that you can open, resize, and save one image in only 30 seconds. For 100 photos, this would take 50 minutes (3,000 seconds), for 500 photos, this would take over 4 hours (15,000 seconds), and for 1,000 photos, this would take over 8 hours (30,000 seconds) of SITTING IN FRONT OF THE COMPUTER REPEATING THE SAME SERIES OF MUNDANE MOUSE AND KEYBOARD ACTIONS NON-STOP! Wow, no wonder he didn't want to do all of this by hand in Photoshop!
So what would have happened if my friend had instead used the
convert big_bunny.jpg -resize 25% little_bunny.jpg convert big_horse.jpg -resize 25% little_horse.jpg convert big_sheep.jpg -resize 25% little_sheep.jpg convert big_goat.jpg -resize 25% little_goat.jpg ...
Let's say that each line takes him 10 seconds to type. For 100 photos,
this would take almost 17 minutes (1,000 seconds), for 500 photos, this
would take 1.5 hours (5,000 seconds), and for 1,000 photos, this would
take 3 hours (10,000 seconds). Sure, it's about 3 times faster than
using a GUI program like Photoshop because he doesn't have to navigate
through menus using the mouse, but it's still mind-numbingly tedious and
repetitive enough that nobody would want to do it! Hmm, but look at
those commands again, and the astute reader will recognize that even
though each of the calls to
Ok, now time for the kicker: you don't need to type out every single command in the terminal for hours on end. You can write a small program called a script to automatically do the typing for you! And herein lies the true power of command-line programs: The power of command-line programs comes from the ability to write scripts that can connect these programs together in order to automate mundane tasks, something that is almost impossible to do in GUI programs due to their inherently interactive nature.
Here is the script that I wrote for my friend to resize his batch of photos:
if [ ! -d resized/ ] then mkdir resized fi for img in *.jpg do convert $img -resize 25% resized/$img done
In English, this script does the following: creates a sub-directory
This script was written in a programming language called BASH, but the choice of
language is irrelevant ... the key power of scripting comes from the
fact that it can leverage the power of existing command-line programs
(in this case, the
for img in *.jpg do convert $img -resize 25% resized/$img done
What this block of code tells the computer to do is to find all the
files in the current directory with the .jpg extension (these are JPEG
image files), and for each file found, execute the program sandwiched
convert one.jpg -resize 25% resized/one.jpg convert two.jpg -resize 25% resized/two.jpg convert three.jpg -resize 25% resized/three.jpg convert four.jpg -resize 25% resized/four.jpg convert five.jpg -resize 25% resized/five.jpg
Cool, eh? The 'for loop' runs 5 times because there are 5 JPEG files,
and each time it runs, it substitutes the name of one of the files where
It took me 5 minutes to write and test that script, and when my friend runs it on his directory of 500 photos, it will save him over 4 hours of mundane clicking and typing to perform the equivalent resizing task on a GUI program such as Photoshop. Now THAT is an enormous improvement in productivity. He can actually spend those 4 hours doing something fun with his life rather than mimicking a machine. He can let the computer do all the boring work!
The enormous setup cost of scripting
Sounds great! So why doesn't everyone use command-line programs and write scripts to automate these mundane tasks and save themselves countless hours of time? Well, I think the reason why is there is an ENORMOUS setup cost that must be paid before people can gain the productivity benefits of scripting. Here are some components of this setup cost:
No silver bullet
All that I have written so far should be obvious and unsurprising for anyone with a decent amount of programming experience. Scripting is a powerful tool to automate mundane tasks. Duh! Programmers have already paid the aforementioned setup cost by familiarizing themselves with a command-line environment and programming over years of work experience. But the challenge is how to lower this enormous setup cost that needs to be paid by ordinary computer users before they can take advantage of the enormous productivity benefits of scripting. There is ongoing research in topics such as 'end-user programming' that attempt to bridge this gap between obscure command-line and user-friendly GUI programs.
Unfortunately, I still see no easy solution in sight, no silver bullet to shatter this barrier to entry. I don't expect my parents or non-programmer friends to be able to easily automate mundane tasks on the computer, and as far as I can tell, they are still going to have to spend hours repeatedly clicking and dragging until they get fed up with their computers and punch a hole through the monitor. Even more pessimistically, I don't think that most people are even aware that these tasks can be automated, so they don't even know that they can ask experts for help, simply because they have not been exposed to ideas from computer programming.
My hope for the future of programming education
I personally didn't write my first scripts until my senior year of college, four years after I had started programming for school and work, even though I was a Computer Science major! And why not? Because nobody taught me about the power of scripting in any class I had taken, so I was simply unaware of the possibilities and thus unable and unwilling to pay that enormous setup cost. Most people, myself included, learn to program in an isolated, sanitized academic environment where they use programming to solve contrived toy problems presented for the sake of pedagogy. Students learn how to calculate Fibonacci numbers, to sort lists of names, and to calculate simple statistics (all great exercises for learning about algorithms and programming techniques, but totally useless in real life), but they don't learn to use scripting to perform useful computing tasks that they could apply outside of the classroom. In other words, students learn programming solely as a professional skill to use in the office, not as a means of assisting them with their own computing tasks at home. One of my hopes for higher education is that programming courses start teaching scripting as part of the curriculum, with an emphasis on real-world productivity-enhancing applications such as the solution to my friend's problem: how to quickly resize a large batch of hundreds of photos.