Book Review: Clojure Web Development Essentials by Ryan Baldwin

A few weeks ago, I got an email from Gavin Cardoza at Packt Publishing offering me a free e-book in exchange for writing a review of Packt's recently-published Clojure Web Development Essentials, by Ryan Baldwin. I think I'll turn down the offer of a free e-book, but I'm happy to write a review because I think Baldwin has done an excellent job on the book. (Disclaimer: I did get a review copy of the book at no charge).

Rather than trying to survey all the tools and libraries available to the Clojure web developer, the author has made the very sensible decision to zero in on Luminus, a mature and well-crafted base project that provides you with the fully-functioning core of a web application, without locking you into a rigid framework. Starting from the Luminus defaults, Baldwin then walks you step by step through the process of configuring the server, configuring the logs (including the tricky bits!), handling forms, persisting data to a SQL database, handling logins, security, and sessions, and finally deploying to production. At each step, you have a fully operational (though incomplete) web application to play with, so the feedback is immediate and satisfying.

In keeping with the Essentials theme, the author sticks to the basics, and resists the temptation to drag in all the interesting related topics that so often muddy and confuse introductory tutorials. Anywhere Luminus offers the developer a wide range of choices, Baldwin picks a good one and runs with it, without falling down the rabbit hole of trying to consider the strengths and weaknesses of each option. This makes the book much easier to follow, retain, and apply, and keeps it from getting boring. Those other choices are out there, and you can learn them later if you're interested, but when you do, you'll already have the basic essentials down, and will be much better equipped to relate new information to what you already know.

As a web developer who uses exclusively MySQL (and/or MariaDB) in my day job, I was interested to see that Baldwin chose to focus on PostgreSQL as the database for this book. That's a good choice for anyone interested in deploying to Heroku or a similar platform, where Postgres services are easier to find than MySQL services, but it might surprise you if you're coming to this book from other web development environments like me. If you haven't played with Postgres before, this book might whet your appetite for more.

If I had to criticize this book for anything, it would be regarding two points. First of all, this book is written from the point of view of a developer working either in OS X or Linux, not Windows. That's understandable, since Clojure development involves a certain amount of command-line work, and Windows brings in a bit more complexity when it comes to getting everything up and running. The omission does make it easier for the book to stay focused on the Essentials of Clojure web development, but might prove frustrating to Windows-based developers trying to get their feet wet in Clojure for the first time.

My other complaint is that the book does not spend a lot of time on the topic of web security. This, again, is probably a tactical decision in the interests of keeping to the Essentials, but being the naturally paranoid person that I am, I wish the book had devoted more discussion to the topic. It does at least walk through the process of using bcrypt to secure passwords, instead of making the mistake of storing them in a decryptable form, so that's a big point in its favor. But an appendix, or at least a link or two, would not have been amiss.

Other than those two complaints, I was very impressed by this book. The writing style is lively and interesting, yet clear and to the point. The examples are easy to follow and do a good job of conveying the technology they are trying to explain. And I love his decision to implement his database interactions using YeSQL rather than a more complicated ORM or ORM-ish system. As someone who has to wrestle with ORM syntax every day (in PHP, no less), the simplicity of straight SQL-as-a-function is something I drool over.

I've got a number of Packt books and I keep going back because they're generally high quality and their e-book format is just right for the Kindle app on my iPad, but I have to say that Clojure Web Development Essentials looks like one of their best so far. Kudos to Ryan Baldwin on a great job, and I hope he decides to do a follow-up book on securing Clojure web applications. (Hint, hint.)

Test-Assisted Design

I'm re-thinking my opinions about unit tests and test-driven design. On the one hand, part of me immediately liked the idea of TDD as soon as I heard it: by writing out your tests first, you have a clear, unambiguous spec for what you're supposed to build. It's clear, because it has to be a test that the machine can run, and it's unambiguous because it's a test. It either passes or it doesn't.

On the other hand, I've been having second thoughts about unit tests ever since I saw Rich Hickey's talk on Simple Made Easy, the Magna Carta of functional programmers. Tests are good, but-- Tests are necessary, but--

The thing is, unit tests could be thought of as a kind of code smell, or a kind of technical debt. You're writing tests because you don't know what your code is going to do. And maybe that's unavoidable. Maybe programming is just too complicated for us to write code that always does exactly what we expect it to do. But still, a code smell, a sign of an undesirable circumstance that we should be trying to avoid or reduce.

Nil is not enough

Here's the context: I'm separating out all my data-store-related functions into their own namespace. Specifically, I'm working on my User model. I want to write a function that creates a new user in my data store, but only if there's not already a user with the same username. No problem, right? If the user already exists, I just return nil, the same as for any other error.

Now I'm using the create-user function I wrote above, and I pass in a user name, and I get back nil. Does that mean there was a database error (not enough space left on drive) or does that mean there was already a user with the same name? I want to get additional information out of nil, but nil doesn't contain any information. It's nil!

Setting up Friend: a few preliminaries

[Revised: found out my first diagnosis and fix were incorrect, but eventually I tracked down the real answers.]

For my summer project, I'm developing a web site that local food banks can use to let people know what items they currently need. Long term, I want this to be accessible via smart phones, but for now I'm just getting started with the basics. I'm using the Luminus framework to kick things off since it's so convenient and well-documented.

I initially set up my site with support for DailyCred, which I only heard about because it was one of the template options for creating a new Luminus project. It seemed pretty easy to set up, and I got as far as redirecting users to DC and getting back the token, but at that point DC would complain that I was using an unknown client ID (even though it was a valid ID that worked correctly in their sandbox), and I'd be dead in the water. Finally I decided to fall back to a more conventional approach: today's project is to rip out all the DC stuff and replace it with Friend. Partly this was frustration with DC, and partly it was because I just got done watching this video, and wanted to take Friend for a spin.

Ah, a secret gateway into ClojureScript

I've been very interested in ClojureScript since it was first announced, but somehow I've never quite made it over that first hill. I think that may be about to change though:

ClojureScript One is a helpful resource for getting started with ClojureScript. It is not a library or a framework. It is an example with lots of documentation. We hope that by looking through this example and reading the documentation we can remove the barriers that are stopping you from experiencing how much fun ClojureScript development can be. Once you have this experience, we think you will be hooked.

You can find ClojureScript One on the web at http://clojurescriptone.com.

 I've downloaded the git repo and got the "Getting Started" stuff up and running in no time. Now if you'll excuse me, I have a REPL or two to play with...

The perfect language

I was browsing through Planet Clojure the other day and saw a post about "The Book the World Needs." Not to give away the plot, but the missing book is an introduction to functional programming for non-FP programmers.

I'm not the guy to write that book, but reading that post reminded me of some of my early programming experiences, and helped me understand what it is that I find so appealing about Clojure.

What I've learned from Salty

I spent the weekend hacking at my Salty library for Clojure. Salty is a thin wrapper around the Selenium WebDriver framework, and it allows you to programmatically control a web browser like Firefox, as I've mentioned before. It turns out there is already a Clojure lib for WebDriver---clj-webdriver---that is full-featured, easy to use, and very mature. For that reason, I'm probably not going to invest too much more effort in Salty. But that's fine: I mostly wanted to try this as a learning exercise, and clj-webdriver makes it easier for me to check my homework. Here's a few notes on what I think I got right and what I think I got wrong.

Re-formatting variable names

(Series: From PHP to Clojure)

Here's a simple problem: as part of my Salty lib, I want a function that will take camel-case variable names, as used by Java, and convert them to the dash format that's idiomatic in Clojure. In other words, I want a function that will take the string "someVariableName" (camel case)and return "some-variable-name" (with dashes).

Salty: a Clojure wrapper for the Selenium Java WebDriver

After my last post, I thought it might be fun to make a full-blown clojure library for working with Selenium WebDriver. I want to do a lot of Compojure/Noir development, and I can see where it might be handy to automate logging in and clicking things with Firefox and IE.

Just to get started, I've set up a repository on github, and have written one quick and dirty test function.



It's not really useful, it's just a quick spot-check you can run at the REPL to make sure everything's set up right. Make sure salty is in your classpath, and then, at the REPL, type (salty.impl/test-with-google). You should see Firefox start up, open up the main Google page, search for "clojure", and then quit. The output in your REPL should be:

Original page title is  Google
Page title after searching is  clojure - Google Search
nil

More to come...

Clojure and the Firefox Zombie

Since it's Halloween, I thought it might be fun to use Clojure to turn Firefox into a mindless zombie, slavishly obeying my every command. Muahahaha.

A simple API

I've been watching Rich Hickey's talk on simplicity, and trying to take it to heart. He's right: easy is easy, and simple is hard. It takes a lot of work, and careful use of the right tools, to end up with "simple."

So here's one tool (dare I say pattern?) that might be useful in building a simple API. I'm building on some posts and IRC conversations that I've run across, so most of this is not original with me, but I'm writing it down here for future reference.

In the spirit of classic "design pattern" methodology, here is the scenario we're trying to address. We need to write some functions that refer to some kind of state. The specific example I'm using is connecting to an IMAP server: I want to establish a connection, grab some message headers, selectively grab the message contents, and so on. Obviously, I don't want to negotiate a separate connection for each and ever IMAP operation I write code for. I want to connect once, and then share that connection with each of the functions that needs to use it.

Keeping up with starting out

There seems to be an impedance mismatch between the frequency with which I make posts about the easiest way to get started with Clojure, and the release of even easier ways to get started, so I'm going to just bag that thread. I'll just summarize my advice in a few simple bullet points:

• If you have to choose between adding clojure to an existing environment that's already heavily customized, and creating a fresh/clean environment via VirtualBox or some other equivalent, I recommend the fresh/clean environment. If you have a reasonably decent processor, it should be responsive enough to be useful, and the advice you find on the web will be a lot easier to apply if your OS is clean.
• If you don't know emacs, it's worth learning just to get to use paredit.
• If you like GUI IDE's, Eclipse + CounterClockwise works well.

Feel free to add any other tips in the comments.

Spring Security on Compojure

I've moved away from using Spring Security for my Compojure/Noir-based web app, but I did eventually get an answer to my question on Stack Overflow, so if anyone's interested, they can look up the example in the answer.

http://stackoverflow.com/questions/6901210/compojure-noir-using-spring-security-for-auth-auth

Using Spring Security with Clojure/Compojure/Noir: Preliminaries

I don't know how to use Spring Security with Clojure web apps, but I'd like to learn how. I've got some clues, and I'm going to try and track them down and see how far I can get.

The first obstacle to overcome is that there doesn't seem to be anything in clojars I can just grab. Fortunately leiningen prints out some detailed error messages, which I can use to manually install the jar files I need. First I'll grab the jar files from http://www.springsource.com/download/community. Just the 3.0.5 release is all I need. I unzip the file, which gives me a directory with a number of different jar files in it.

Next, I cheat by editing my project.clj file to ask for a jar that doesn't exist. The lein deps command chugs along for a while, and then spews out a huge error dump, but with a little digging I can find this message somewhere in the middle:

Unable to resolve artifact: Missing:
----------
1) org.springframework.security:spring-security-parent:jar:3.0.5.RELEASE

  Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-parent -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=/path/to/file

There's the magical incantation I need. I'll just cheat again by editing the suggested maven command so that it installs each of the jar files that I downloaded from springsource. I don't know yet which specific jar files I'll need, so I'll install them all.

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-acl -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-acl-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-aspects -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-aspects-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-cas-client -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-cas-client-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-config -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-config-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-core -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-core-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-ldap -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-ldap-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-openid -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-openid-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-taglibs -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-taglibs-3.0.5.RELEASE.jar

mvn install:install-file -DgroupId=org.springframework.security -DartifactId=spring-security-web -Dversion=3.0.5.RELEASE -Dpackaging=jar -Dfile=./spring-security-web-3.0.5.RELEASE.jar

If I cd into the directory where I extracted the Spring Security jars, I can just cut and paste this code, and have each of the spring-security-* jars installed in my local .m2 directory, where leiningen can find them.

Now I go back and edit project.clj, and add the core spring-security jar to my dependencies:

(defproject mm2 "0.1.0-SNAPSHOT"
            :description "FIXME: write this!"
            :dependencies [[org.clojure/clojure "1.2.1"]
                           [noir "1.1.0"]
                           ;; ADD SPRING SECURITY HERE:
                           [org.springframework.security/spring-security-core
                            "3.0.5.RELEASE"]]
            :main mm2.server)

Run lein deps and ... yup, the jar file is in lib/. It worked! Now I just need to figure out what other jar files I need (if any) and how to hook them into my clojure project. But that's a project for later.

Setting up Emacs clojure mode

Picking up where we left off in Starting from Scratch, we're next going to set up Emacs to be able to use clojure-mode for editing code, and to compile and run our Clojure programs for us. If you're new to Emacs, don't worry, we'll be taking this one step at a time.

The plain Linux system we installed in the first post comes with Emacs version 23. Version 24 comes with an Emacs extension called package.el< that makes it easier to add new functionality to Emacs by downloading pre-built extensions from sites like Marmalade. Version 23 doesn't have this built in, so we'll have to add it by hand, which fortunately is not hard.

$ mkdir ~/.emacs.d
$ cd ~/.emacs.d
$ wget http://bit.ly/pkg-el23

This will pull down the package.el file and install it in your .emacs.d directory, which is the standard place to put extensions for Emacs. Having the file in the .emacs.d directory only makes it available to Emacs, though. To actually use it, we need to modify the .emacs startup file.

$ emacs ~/.emacs

There may be some code already in the .emacs file, but we can safely skip over it for now. Go to the end of the file and add the following text:

(add-to-list 'load-path "~/.emacs.d")(require 'package) ;; loads package.el so we can use it
(add-to-list 'package-archives  '("marmalade" . "http://marmalade-repo.org/packages"))(package-initialize)

Quit Emacs and then open it up again. You should see the same opening screen as usual, but if there's a typo in the code you added to .emacs, you may see an error message instead. If that happens, open up .emacs again and double-check your typing, particularly single quotes and the dot between "marmalade" and the URL---it's not a comma. (The keyboard shortcut for opening a file is C-x-f, which is shorthand for Control-x followed by Control-f.)

Once your .emacs file loads correctly, we can go on to the next step. We're going to use the Emacs command line, aka the "minibuffer," to actually install a couple of packages. To get to this command line, type M-x, which is shorthand for Meta-x. "Meta-x" means hold down the "meta" key (Alt key on Windows keyboards) while typing an "x". At the very bottom of your Emacs window, you should see the string "M-x" followed by a flashing cursor. Emacs is waiting for your command. (Note: if you mistype and need to get out of the minibuffer, type Ctrl-g.)

We're going to give it a couple of commands. Let's start by installing paredit, a package that will automatically balance our parentheses for us.

M-x package-installInstall package: paredit

Next, we'll install clojure mode, which uses paredit.

M-x package-installInstall package: clojure-mode

Now we'll go back and make one last edit to .emacs. Type the following at the end of the file:

(defun turn-on-paredit () (paredit-mode 1))(add-hook 'clojure-mode-hook 'turn-on-paredit)

This will automatically turn on paredit mode whenever we edit a .clj file. Paredit mode is something people either love or hate because it forces you to keep your parentheses, braces, and square brackets matched. That's good, because otherwise you can get subtle bugs due to parentheses not closing where you thought, but sometimes paredit won't let you delete a single parenthesis even though you just want to move it someplace else.

I'll tell you the paredit trick, though: cut and paste. You can cut and paste any of the parenthesis-like characters even if paredit won't let you backspace over them to delete them. Less convenient than a simple backspace, perhaps, but it's a small price to pay for the extra functionality and safety that paredit gives you, especially if you're new to LISPs and Clojure.

We've got one last command to type at the command prompt:

$ lein plugin install swank-clojure 1.3.2

This installs the code that allows Emacs to hook up to Clojure so that you can actually run a Clojure REPL inside Emacs. The command to start a Clojure REPL inside Emacs is M-x clojure-jack-in. The REPL takes a few minutes to come up -- no getting around that JVM startup time -- but after a little bit you should see the friendly user=> prompt.

And now you're good to go: you have a high-powered editing environment all hooked into the Clojure REPL, so you can edit code, and run it, and even edit it while you are running it. How cool is that?

Comments, criticisms, corrections and questions are all welcome. Feel free to copy any or all of this post to use and/or improve for your own blog.

Starting from scratch

I've been wanting to write about Clojure for some time now, and I've decided to do so from the perspective of a PHP programmer (that's me) making the transition to functional programming, using Clojure. I like to start at the beginning, so in this post, I'm going to set up a Clojure programming environment from scratch.

[NOTE: Early feedback on this post suggests that I may be making things look harder than they really are. Clojure works just fine in any environment that runs Java, and can be developed using any editor or IDE. My goal in starting from scratch is purely didactic. I'm starting with a simple, bare-bones environment so that anyone who wants to follow along can easily do so without being distracted by issues like "I played with emacs a while ago and my .emacs file has some cruft in it" or "My classpath seems to be pointing at some old jar files I installed a couple years ago."

If you're not interested in setting up a clean, minimal environment, feel free to skip ahead to the "Installing leiningen" section.]

I didn't think that would work.

Just saw a quick code snippet on the #clojure IRC channel. I didn't believe it would work at first, but apparently it does.

=> (defmacro rnd [] (rand-int 10))
#'user/rnd
=> (defn foo [] (rnd))
#'user/foo
=> (foo)
8
=> (foo)
8
=> (foo)
8

So every time you use your macro, it expands to a random number *at compile time*. The foo function is thus defined as a constant value, but it will be a different constant every time you re-load the code and run it.
Not sure what that's useful for, but I bet it could do something cool.