Married

I'm back online after three weeks vacation. Purpose of said vacation was to get married and go on honeymoon. Only thing to say about that is: huge success. I'm very, very happy.

While I've been gone, Ubuntu has had its developer summit and decreed that Ubuntu 12.10 will have only Python 3 on the CD. As such, a few people have done a bunch of work on the testtools cluster (fixtures, testscenarios, etc.), making them Python 3 compatible, or fixing compatibility bugs where they existed. It's such a nice feeling to come back from vacation and taste the delicious fruits of other's labour. Well done all!

Local Juju

I have a bit of a fetish for being able to hack offline. It's becoming increasingly old-fashioned, I know, but I'd like to think that eventually it will become a quirky and charming eccentricity, such as one might find in a British amateur sleuth.

I wanted to get up and running with Juju, Canonical's funky cloud orchestration thingy. We're doing a lot more service-oriented stuff in recent months, and it's very handy to be to just deploy something without having to faff about for a couple of hours. Since I'd like to stand a chance of doing this without a net connection, I set it up locally.

With the help of the Charm School page, the local provider docs and the kind folk at #juju on Freenode, I got something up and running. Some notes about the experience:

• I did all of this in precise, and it seemed to work just fine. I substituted 'precise' for 'oneiric' wherever I saw it.
• I had to manually add my user to the libvirtd group. The instructions implied that this was not necessary.
• juju bootstrap finished very quickly for me and without any actual evidence that it had debootstrapped the whole OS.
• When I deployed services, I didn't get any feedback that anything was going on. The recommended debug-log command showed very little output for a while. My guess is that this was while packages were downloading.

All in all, pretty impressive. I'm really looking forward to mucking around with this a bit more

Bad Snake Joke

Just between you and me, I'm getting a bit bored of writing Python all the time. It's a good language: I can write code that's functional or OO as the case may be; it has lots of libraries; it has Twisted, which is incredibly useful; it's not too hard to make a big project and keep it clean. It's the least awful programming language that I know of. That said, I'd like to do something different.

I would like to play with something makes static typing rock (like Haskell), or that has what Rich Hickey calls "polymorphism a la carte" (again, like Haskell, or Clojure). I would like to make something that stands a chance of feeling snappy (perhaps PyPy?), a chance to use something with richer debugging and refactoring tools, or to get a feel for doing some serious concurrency work outside of an event loop.

But mainly, I'm just bored of Python.

Also, I increasingly suspect that Python doesn't have legs. The Python 3 language change has increased the split between the core developer community and people writing code in the field. Python continues to be slow, both with start-up time and while running. The only people who seem to care are the PyPy developers, but I doubt I'll ever get to use it for commercial development. It's tricky to write code in Python that takes advantage of multiple cores, and even my phone has multiple cores now.

When I get the chance, I'm going to do more with Haskell and Clojure in my spare time. I'm not sure if there is something less "fringe" that I could recommend for use at Canonical. Go is a possibility, if a slightly disappointing one (interfaces are cool, but why oh why didn't they do typeclasses?).

testtools 0.9.14 released

testtools's sister project, subunit, was using a private API that we deleted in the 0.9.13 release.  This release restores that API in order to smooth out the upgrade path.

If you don't use subunit, then this release won't matter very much to you.

testtools 0.9.13 released

It has been a while, but testtools 0.9.13 is finally out! Lots more matchers and bug fixes, as well as improved error reporting. Full release notes on Launchpad.

Thanks to James Westby, Graham Binns, Francesco Banconi and Robert Collins for making this release possible.

Simple made easy

Rich Hickey did a great talk at Strange Loop called "Simple Made Easy". You should watch it.

When I tried to explain the talk to someone, I stumbled a lot and it was obvious to me that I didn't really understand it. So I'm going through it again and turning it into a blog post, purely for my own gain.

This is roughly the first half of the talk. Not much of my own analysis or opinion is inserted, and I've pretty much stuck with Hickey's illustrations and phrasings. Thus this post is pretty derivative. Oops.

Simple vs Easy

"Simple" means one thing, "easy" another. Simple is the opposite of complex. A thing is simple if it has no interleaving, if it has one purpose, one concept, one dimension, one task. Being simple does not imply one instance or one operation: it's about interleaving, not cardinality. Importantly, this means that simplicity is objective.

Easy is the opposite of hard, or difficult. A thing is easy if it's near to hand, if it's easy to get at (location), if it's near to our understanding (familiarity) or skill set or if it's within our capabilities. This means that ease is relative.

Speaking English is dead easy for me, but that doesn't mean that speaking English is intrinsically simple. I find French quite difficult. Little French children speak French all the time, and there's always a part of me that thinks, "Boy, those kids are clever, being able to speak a foreign language at that age", but that's silly. It's easy for them, it lies near to them.

This distinction between simple & easy is good one, and is useful in all sorts of areas. But how does it relate to software?

Constructs vs Artefacts

As programmers, when we make software we are working on constructs: source code, libraries, language concepts and so forth. Rich contends that we focus on the ease of use of those constructs: How many lines of code? How much boilerplate? Will new developers be familiar with our technology?

But all of this is secondary. What actually matters is the artefact, the running programs that users actually use. Does it do what it's supposed to do? Does it do it well? Can we rely on it working well? Can we fix problems when they occur? Can we change it? You know, the interesting problems.

Thus we need to be assessing our constructs – our code, our technology choices – based on the attributes of the artefacts that we'll create, not based on the experience of typing code in.

Limits

We can't make something reliable if we don't understand it. And, actually, everyone's understanding is pretty limited. We can all only hold a small number of things in our head at once.

When things are complex, many parts are tied together by definition. You can't pull out just one piece and consider it because it's intertwined with other pieces. This creates an extra burden to understanding a system and thus makes it difficult to reason about the system.

You do need to reason about a system, both to know what to change and to be able to do so without introducing defects. Tests, refactoring, rapid deployment and all that are great, but to make a change to the system safely & without fear still requires you to be able to reason about it. Every bug in your product that was found in the field passed the type checker and passed all of the tests.  Your type system doesn't tell you what change to make next in order to get the software you want any more than guard rails on a highway tell you how to get to Grandma's.

Speed

Focusing on ease and ignoring simplicity means that you'll go really fast in the beginning, but will become slower and slower as the complexity builds.

Focusing on simplicity will mean that you'll go slower in the beginning, because you'll have to do some work to simplify the problem space, but making sure that you only have intrinsic complexity means that your rate of development will remain at a high constant.

There are no actual numbers for this.

Complicating constructs

Many complicating constructs are available, familiar, succinctly described and easy to use. But none of that matters to end users. What matters is the complexity they yield. This complexity is incidental, it's not intrinsic to the problem.

If we build things simply, then the resulting system is easier to debug, easier to change and easier to understand.

Compare a knitted castle to a castle made of Lego. The knitted castle might have been great fun to make, and might have been really easy if knitted using a loom and cutting edge knitting tools, but there's no way that it's easier to change than a Lego castle. It's not about the ease of construction, it's about the simplicity of the artefact.

How can we make software easier?

Well, we can install it to make it easier by location. We can learn it and try it to make it easier by familiarity. We can't do much about our capabilities though. If we want software to be easier to comprehend, we are going to have to bring it down to our level. We have to make it simpler.


Take Lisp as an example. It's hard for many people because they don't have a Lisp installed, or their editor doesn't support paren matching, but they can make it easier by installing a Lisp and getting a plugin for their editor. It's also hard because it's unfamiliar. Who'd have thought that parens could go on that side of the function? But you can gain that familiarity quickly enough.

But parens in Lisp are used for functions and for grouping data. That's hard to get your head around, and that's because it's complex. It braids together two distinct notions.

Undistract me

Here's a thing that happens a lot to me: I'm doing some work, and as part of that work I need to run a command in my terminal that takes a little while. I run the command, look at it for about a second and then switch to doing something else – checking email, perhaps. I get deeply involved in my email checking, and then about twenty minutes later I switch back to the terminal and see the command has finished. For all I know, it finished nineteen minutes ago, and I was just too engrossed to notice it.

This is a big productivity sink for me, especially if the command happened to fail and need retrying. I'm not disciplined enough to just sit and watch the command, and I'm not prescient enough to add something to each invocation telling me when a command is done. What I want is something that alerts me whenever long running commands finish.

Well, that thing now exists, thanks to glyph's script that provides precmd and postcmd support to bash and a lot of help from Chris Jones of Terminator.

To use it right now:
 $ bzr co lp:~jml/+junk/shell-tools
 $ . shell-tools/long-running.bash
 $ notify_when_long_running_commands_finish_install


You'll see that if you run a command that takes over 30 seconds to complete, it will pop up a notification, which should hopefully take you away from whatever it was you are doing and back to the task at hand.

If you look at the code, you'll see that it installs two hooks: precmd and preexec. preexec runs just before the shell launches a command, and precmd runs just before it prompts for the next command. Our preexec stores when the command was launched and the precmd checks to see if it finished within a certain time frame. If not, it sends out a notification.

Currently, you'll get a notification when you finish reading a long document, since the command finishes a long time after the command starts. Obviously this isn't ideal. I think the fix is to only send notifications when the shell doesn't have focus. Unfortunately, that's a little tricky and I think is going to be highly terminal specific.

Anyway, I'm a total shell newbie, so I'd love to know if there's any way this could be done better.  Also let me know if you find this useful, or you know of someone who has already done this.