Mere Code

Flow, Interruptions and Gold-Titanium Alloys

2008-08-18T13:21:00.005+10:00

As I mentioned earlier, I've just had a very productive and fun weekend of hacking on testresources. But why was it so good?

Fast Commits

Because I was working offline using Bazaar, I could make commits to testresources and they'd be done before I could Alt-Tab back to Emacs.

Fast Tests

It takes about 0.15 seconds to run the test suite. This meant I was running it all the time, which in turn meant that I was totally certain about whether the code worked all the time.

Clarity of Purpose

I knew where I was and where I wanted to go.

No Rabbit Holes

The testresources code base is small enough that it is nearly impossible to get distracted by other systemic defects. If there is such a defect, it's probably the one that you're working on right now. This might be another way of saying...

No Blockers

I wasn't waiting on anyone to do anything. There weren't any other bugs that needed to get fixed before I could continue.

No Integration

The system is about as self-contained as it gets. No signals, subprocesses or sockets: just Python. The only dependency is pyunit3k, which is small, robust, well-tested and also pretty darn self-contained.

No Interruptions

I wasn't answering questions on IRC, taking Skype calls, handling incoming email, doing the washing, checking Facebook, catching up on my blogs, watching a film or wrestling a bear. "Fast Commits" and "Fast Tests" probably fall into this category too.

In other words:

There is nothing except this. There's no art opening, no charity, nothing to sign. There's the next mission, and nothing else.

testresources: fresh blood

2008-08-18T11:11:00.004+10:00

Last weekend I spent a bunch of timing hacking on testresources with Rob over at Chris's place. testresources is an extension to unittest that allows tests to specify the resources they use declaratively, so that these resources can be cleanly shared between tests. On Saturday, we talked a bit about direction and decided on some goals. On Sunday, I got busy.

The idea isn't particularly new: zope.testing has had "layers" for a long time. The key differences between testresources and Zope layers are:

You can use more than one TestResource. You can only use one layer. Allowing tests to specify many resources makes it easier to use only the resources that you need, which in turn makes for a faster test suite.
A test that uses testresources can be used independently of other testresources machinery. With layers, you pretty much need the whole shebang. This means that you can use testresources with trial, bzr, nose, tribunal or unittest.TextTestRunner.
Layers have magical inheritance stuff. testresources has no magic.
Layers do some work in subprocesses to accommodate some setUps that can have no tearDown. testresources doesn't know what a process is.

That said, I'm mainly familiar with layers from Zope 3.2—they may have changed since then.

testresources itself is somewhat old. Rob hacked up the first version about three years ago and little has happened to it since. I've always agreed with the approach, but have also had a few qualms about implementation details that made me reluctant to use it or to recommend it. Now, it's well on its way to being something that I can trust and perhaps, love.

If you are interested, check out my branch or at least flick through the TODO file.

Cool Launchpad/Bazaar hack

2008-07-25T15:38:00.002+10:00

If you push a lot of Bazaar branches to Launchpad, you might want to add something like this to your .bazaar/locations.conf:

[/home/jml/Code]
public_branch = lp:~jml
public_branch:policy = appendpath
push_location = lp:~jml
push_location:policy = appendpath

Once it's there, 'bzr push' will Just Work... as long as you want it to do something like:

jml@rhino:~$ cd ~/Code/pyunit3k/trunk
jml@rhino:~/Code/pyunit3k/trunk$ bzr push
Using saved location: lp:~jml/pyunit3k/trunk
No new revisions to push.

So if I've got a branch in ~/Code/<project-name>/<branch-name>, it'll be published at https://launchpad.net/~jml/<project-name>/<branch-name>. This is only a default, of course: you can override it for a given branch by doing:

bzr push --remember <new_location>

Or you can override it for a bunch of branches by hacking locations.conf.

Neat huh?

I Love Data

2008-07-07T17:00:00.003+10:00

At work, we recently started to graph the time it takes to access a branch in Launchpad via SSH, both on our staging and production servers.

I love this so much. Having this data is liberating. It's like turning on a light in a dark room: suddenly I can go from uncertain, careful, hesitant steps to bold, confident strides.

In fact, the metaphor stretches further. This new light has revealed objects of interest (that is, spikes in the time taken) that I wasn't aware of before. I need to know how long these spikes last, how they correlate with load on the system and so forth. Soon there'll be more graphs, and I'll be able to correlate them and analyze them and suck on their delicious, numerical marrow (the light metaphor long abandoned).

Regardless of whether it's liberating illumination or nourishing, savoury meat, this new graph makes me wonder why I don't chart other things I care about. Having such graphs would help me see the impact of my actions. I could chart things like my bank balance, my waistline or the number of Latin words I learned this week. Then I could answer questions like "do I spend more on Tuesdays?" or "can I afford this iPod/cake/gerund?". This is important, because when I am tempted with a sleek/delicious/perplexing iPod/cake/gerund, I fall back on my own judgment. I think it's time to confess, dear reader, that my own judgment isn't very good. And yet I continue to trust it.

The reason I don't have such graphs is that they are inconvenient to maintain. A graph of branch access times is easy. All you need to do is describe how to get the measurement, and then do a bit of once-off set up. If you want to graph your body weight, you need to get on the scales at a fixed time and then look at the dial and adjust for parallax and then write a number down somewhere and maybe note down whether this is before or after a meal and then take the number and then add it to a spreadsheet. Most online banking sites I've seen are even less convenient than this.

Getting data on life is hard, but for programming it's easy. What things do you care about on your project? Features, user experience, how fast bugs get fixed? Can you quantify these things? Can you make a pretty picture out of it? If so, do it now! Post your answers here, set up something like Cricket and then profit!

P.S. I can't believe I got to the end of this post without saying how awesome the Canonical sysadmins are. Let me correct that now. The Canonical sysadmins are heck of awesome. They could simply walk into Mortor.

Somebody Do This Please

2008-07-03T16:17:00.004+10:00

<jml> What's the difference between Gobby and a wiki?
<mwh> NAT

GMail feature request

2008-07-01T09:09:00.000+10:00

First, where should I make these requests?

Second, I'd like to be able to open my GMail without seeing all the new mail in my inbox. Very often, I open up GMail just to look up some fact that I've forgotten, or to put a date to a particular event. Seeing new mail and an unempty inbox distracts me, leading me down a rabbit hole of processing.

I guess a GreaseMonkey hack wouldn't be too hard.

2008

2008-06-23T20:12:00.000+10:00

Firefox 3 is awesome!

Really Satisfying

2008-06-13T15:23:00.000+10:00

I have just returned from the shops with a Snickers bar. The packet says that one out of every six Snickers bars will instantly win me a free Snickers bar.

So, statistically speaking, how many Snickers bars do I have? Show working.

Aren't code reviews great?

2008-06-13T14:37:00.000+10:00

Ben Sussman-Collins writes about programmer insecurity and how a lot of programmers dread having others see their code.

Some of the best programmers I've worked with are in love with code reviews, and all great writers have editors. (The "linked list" is in no way exhaustive.) Personally, I see reviews as being a great opportunity to learn.

Sussman-Collins also writes about how distributed version control can exacerbate the "isolated genius" problem by shielding ones code from the public eye. This is definitely a big potential drawback of DVCS, but one that's largely mitigated by a culture of short-lived branches and an easy-to-use public registry of all branches for a project.

Another Bazaar story

2008-06-13T11:29:00.000+10:00

I'm currently hacking away on Launchpad's support for "stacked branches", something that will really make Launchpad's codehosting a joy to use.

At the moment, I'm writing some tests that require a user to login. This was becoming a bit cumbersome, until I remembered something: Tim has recently landed some code to make this easy. But how do I get these changes without messing up all of the work I'm doing now? bzr shelve to the rescue!

'shelve' interactively goes through each change you've made to your current working tree and allows you to decide whether to keep a change or shelve it. It comes with a twin command 'unshelve', which lets you interactively restore your changes.

In this case, I don't even care about the interactivity, so here's what I did:


# Shelve my changes
bzr shelve --all
# Fetch the latest trunk
cd ../trunk
bzr pull
# Merge it into my branch
cd ../stacking
bzr merge ../trunk
bzr ci -m "Merge in changes from trunk to get login testing improvements."
# Restore my changes
bzr unshelve --all

The 'shelve' command comes with the bzrtools plugin, and I am basically in love with it.

As a parting shot, I should mention that things like bzrtools aren't accidents. They are natural and inevitable when you have good APIs in a high-level language and a very nice plugin system.

And now I'm off to keep working on this branch.

Real Life

2008-06-05T11:17:00.000+10:00

Inspired by Mikey, I've set up my own real-life blog.

I've done this one using Blogger's "upload to remote server" feature. It looks nice, has the features I need and means less WordPress & PHP. I'm hoping to migrate this sucker to the same technology.

Neat Bazaar feature

2008-06-02T13:28:00.001+10:00

Ever find yourself working away on a branch, enjoying yourself and getting just a little carried away? Maybe you're working on a feature and you notice and fix a bug that's not strictly related to that feature.

If you catch yourself in time, there's a nice little feature in Bazaar that can help with this: bzr merge --uncommitted. It will merge in the changes that you've made to your working tree but haven't committed yet.

e.g.


$ cd some-feature-branch

... hack hack hack ... oops!

$ cd ..

$ bzr branch trunk bug-fix-2357

$ cd bug-fix-2357

$ bzr merge --uncommitted ../some-feature-branch

$ bzr ci -m "Fix up bug 2357. Found this while working on some-feature."

$ bzr send

Don't know what bzr send does? Trust me, you want to find out.

What I meant

2008-04-21T11:39:00.000+10:00

z3p recently blogged about a comment I made in a code review. In the review, I linked to:

That's a negative and grumpy way of phrasing an idea that I've come to value a lot: good code expresses its intent clearly.

When looking at a patch, the reviewer needs to understand two things: the intent of the code and the intent of each change to the code. To be clear on the former, you need:

intent-revealing names.

good abstractions / interfaces.

good, small tests.

simple implementations where possible.[1]

docstrings where appropriate

comments where appropriate.

That's not exhaustive, but it's in a rough order.

To be clear on the intent of your change to code, you need:

Small patches.

A good bug / spec with a good, short summary.

A review request letter, summarizing your implementation strategy, any compromises you made, gaps in testing, future work etc.

That's not exhaustive either. In #2037, I didn't understand the motivation for lots of the code, nor for some of the changes to the code.

I'm indebted to Andrew Bennetts for teaching me that the first duty of a reviewer is to ensure that the code is clear and to Tom Berger for reminding me that compromises are worth noting.

----

[1] Actually, this reminds me of something I heard a preacher say, "before I give a sermon, I go through it, find everything clever, and take it out" (I paraphrase, not having a reference on hand).

In as much as sermons and code should both be ego-free communications of ideas, I think this is sound advice for hackers.

bzr-removable

2008-04-21T11:04:00.000+10:00

The plugin I talked about in a previous post is now at https://launchpad.net/bzr-removable. Please file bugs.

Thanks to Michael Hudson and Daniel Watkins for submitting patches.

You can get the plugin by typing 'bzr branch lp:bzr-removable' on the command line.

OK I lied

2008-03-18T14:36:00.000+11:00

The next post is actually about a Bazaar plugin that I've now got ready to share. To get it, 'bzr branch lp:~jml/+junk/merged-branches'.

Once you've got it, run 'bzr merged-branches' in the trunk of your project. It will then show you all branches in sibling directories that are safe to delete. That is,

They have no uncommitted changes.

They have no "unknown" files. (Files outside of version control that haven't been explicitly ignored.)

They have no shelved changes. The plugin will only check for this if it can find bzrtools.

The branch is now at the point where it works for me, but it still belongs in '+junk' — here's why:

It assumes that you have a trunk branch in the same directory as all of your other branches.

It assumes that branches and working trees are the same thing.

It's called 'merged-branches' when it really means 'safe-to-delete'.

It doesn't make it easy to see why a branch is not safe to delete.

It doesn't let you customize the conditions of the search. Maybe you want to see all branches with uncommitted changes.

Still, if you are like me and make a lot of branches, it's quite useful. I'll tolerate bugs, accept patches and welcome encouragement.

User Experience — When Reality Attacks

2008-03-13T10:24:00.000+11:00

I've just got back from a hectic week in London, where members of the Bazaar community got together and thrashed out a bunch of important topics.

We talked about "user experience" and how we all want Bazaar to be a joy to use. More than one person said that we have been focusing too much on features and performance instead of user experience. The term was never really pinned down, but it's fair to say that there are things other than convenience and speed that affect how users feel while using Bazaar and that we need to work on those things, once we figure out what they are.

I think I might know the name of one of them: errors. Next post: "Notes on error".

Three-ways on Pidgin?

2008-02-18T10:04:00.000+11:00

Dear Lazyweb,

How do I initiate a three-way chat with Pidgin 2.2.1?

jml

What I do all day

2008-02-15T16:57:00.000+11:00

Kick-arse summary here: The great source code supermarket.

The Economist Index

2008-02-11T15:10:00.000+11:00

The Economist occasionally publishes a "Big Mac Index" — how much the world's favorite "burger" costs in each country, translated into a common currency.

Why they don't publish an index of how much an issue of the Economist costs? All of my issues have the price of the newspaper in lots of Asia-Pacific currencies. A small sample:

Australia, 9.39855 USD

Cambodia, 6.00 USD

China, 10.43655 USD

Hong Kong, 7.6917 USD

India, 5.0522 USD

Japan, 10.70535 USD

New Zealand, 8.6669 USD

Can anyone tell me why the price varies so much?

WoW account temporarily suspended

2007-12-10T12:43:00.000+11:00

Howdy Partners,

My World of Warcraft account is temporarily suspended for silly administrative reasons. I'm not in any rush to clear those up, so don't expect to see me on Khaz'goroth or Frostmourne any time soon.

Regards,

jml

Obligatory

2007-11-15T02:02:00.000+11:00

Not using Facebook anymore.

Have fun kids.

Fake Plastic Tests

2007-10-23T12:40:00.000+10:00

Ever wondered what's the difference between a FakeReactor, a StubReactor and a MockReactor? Find out in this brief article on test doubles.

The most retentive thing I've done

2007-10-16T18:15:00.000+10:00

Cracked Mac

2007-10-16T18:09:00.000+10:00

OK, I've started with this whole photography thing, I might as well run with it.

My Mac cracked:

Around like Grover

2007-10-16T18:06:00.000+10:00

What's up hep-cats?

I've been travelling around the world, weaving in and out of timezones like Jason Bourne navigates facial expressions. Last week's adventure took me to tropical Dunedin — the southiest outpost of Canonical Ltd. (I'm trying to get a mention on Language Log.)

Those few Hobartians who keep reading will notice that Dunedin, well, umm... here's a photo:

Pay careful attention to the hills in the background.