Shell scripting with Clojure

We had a nice afterwork Clojure learning session with a colleague the other day during which we quickly hacked together a shell script that reads a directory containing Selenium test result files and outputs all the failed test cases. (Why we haven’t automated the Selenium test running and result reporting with e.g. Jenkins is another story..)

In the session I learned at least about the following:

  • making standalone command-line executable Clojure scripts (as long as you have Leiningen installed) that still use Leiningen to handle the dependencies
  • reading and writing files
  • using Enlive to parse and look up particular data from HTML files

In this post, I’ll describe how to create command-line executable Clojure scripts with lein-exec. In a later post I will describe the test result parsing script we put together.

Command-line executable Clojure scripts with lein-exec

The only prerequisite here is that you have Leiningen installed but it should be pretty safe to assume this as lein seems to be the de facto build tool for Clojure.

Now, when you want to do quick standalone Clojure scripts without having to create a complete lein project, but still want to use lein to manage the dependencies to any 3rd party libraries, there is a nice lein plugin called lein-exec.

The installation is simple (the following assumes Leiningen 2, see the lein-exec readme for 1.x installation instructions). A global installation (meaning you can use lein-exec any time you call lein, rather than specifying it as a project-specific plugin) of lein-exec is done by adding the following to your ~/.lein/profiles.clj (if profiles.clj doesn’t exist, create it first).

;; as of writing this, 0.3.1 was the latest version
{:user {:plugins [[lein-exec "0.3.1"]]}}  

Now we can quickly create and run scripts:

(println (+ 1 2))

running foo.clj:

$ lein exec foo.clj
3

This isn’t that impressive yet, but once you add dependencies to third-party libraries, the usefulness becomes more apparent:

(require 'leiningen.exec)

;; Add a dependency to the classpath on the fly
(leiningen.exec/deps '[[enlive/enlive "1.1.4"]])

(require '[net.cgrand.enlive-html :as html])

;; Grab and print the title element from the Google front page using Enlive
(println (html/select (html/html-resource (java.net.URL. "http://google.com")) [:title]))

and let’s run it:

$ lein exec bar.clj
({:tag :title, :attrs nil, :content (Google)})

Now lein and lein-exec handle the enlive dependency behind the scenes. This is nice as now you have all the power of the Clojure ecosystem available for you even in quick one-off scripts.

It is also possible to make the scripts executable themselves if you are in a *nix environment. You need to download a lein-exec script and save it somewhere in your path. Also, if your lein executable is called something else than `lein`, you have to modify the lein-exec script to reflect that.

$ wget https://raw.github.com/kumarshantanu/lein-exec/master/lein-exec # download
$ chmod a+x lein-exec # make the script executable
$ mv lein-exec ~/bin # move to ~/bin, assuming it is in PATH

Now if you add the following to the first line of your script:

#!/usr/bin/env lein-exec
(println (+ 1 2))

you can execute the script directly (as long as you give it the execute permission):


$ chmod +x foo.clj
$ ./foo.clj
$ 3

That’s it for now. I will follow up with the Selenium test result parsing script soon.

Read more about stand-alone Clojure scripting:

  • A Brief Beginner’s Guide To Clojure – Standalone Scripts
  • UVTC’s Blog – Scripts in Clojure
  • Advertisements

10 comments

  1. I’m all for for scipting with Clojure, except for the load time needed for the JVM. Is this an issue with lein-exec? As in how long does it take to run the script in your example?

    • There is indeed a noticeable delay. I can check the run times later. But in any case, if “low latency” of the script is essential, Clojure is probably not the best choice.

    • You could take a look at Drip (https://github.com/flatland/drip). Basically always keeps a JVM on deck.

      However, for myself, the the reason that I need fast shell execution is due to the fact that I end up having to rerun a bash script about a billion times to fix all the endless mistakes that I make. If I develop my script in repl, then I only have to run it a couple times from the bash shell. Maybe just once! So, the JVM load time doesn’t really matter.

      • That’s right, I hadn’t thought about it but when writing a sh / bash script I continuously run the whole script from the command-line when iterating to get it right. In case of Clojure there is usually no need to run the whole thing as you can develop and test the individual parts (and of course their compositions also) in the REPL..

    • Running the simple foo.clj (+ 1 1) with lein exec takes total of about 0.7 .. 1 seconds with my quite beefy desktop machine. Of course the runtime of the actual code is something like 0.01 milliseconds while the majority is probably due to the JVM start time.

      The Drip project mentioned by Vr looks interesting.

  2. Nice. Thanks for writing this. Most of my apps start out as simple command line scripts. And it is nice to be able to avoid the overhead of creating a lein project to write a simple one-liner.

    I also found that as long as I add {:user {:plugins [[lein-exec “0.3.1”]]}} to ~/.lein/profiles.clj I don’t need to download lein-exec. I can just use “lein exec foo.clj” and “#!/usr/bin/env lein exec” (i.e. without the hyphen). This shortens the installation process a little bit more. I am using this to deploy scripts across a cluster of machines. So a parsimonious install is more desirable.

  3. Correction on the last message: lein-exec is needed for /bin/env.

    Here is what would be really useful. A single bash script that installs lein and lein-exec if they are missing, and then runs the Clojure code as a script. This way I can give the script to people and have them run them without any dependencies.

    I prefer giving out scripts rather than uberjars—because this way if someone wants to modify it it is a lot easier.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s