Lange's Law

"With threads, all things are possible."

Stacked branches in Launchpad

Via the medium of the charming Matthew Revell, I've posted about stacked branches on the Launchpad News blog. I'm so glad this is finally out.

Super Happy?

Does Sydney have a Super Happy Dev House or Hacking Society or anything where a bunch of busy programmers can sit down together and hack?

If not, could you please organize one for me — that'd be great.

More bzr hacking

I've extended bzr-establish to give it a new command: hack.

'bzr hack lp:foo' will create a repository with working trees called 'foo' and fetch the lp:foo branch into that repository.
'bzr hack --repository ~/repos lp:foo' will create a repository in '~/repos/foo', a working tree area in 'foo', fetch the branch into the repo and make a light-weight checkout in the working tree area.

You can also specify a non-Launchpad branch, e.g. 'bzr hack http://example.com/some/branch/trunk foo'. This will make a repository called 'foo', put the branch in there and... well, you get the picture.

The plugin lives at lp:~jml/+junk/bzr-establish. It's buggy, not particularly well tested and rough as guts. Still, it's worth a play.

Bazaar hacking

I spent a little bit of time mucking around with some Bazaar plugin ideas I've had.

The first is a command that switches from "branches and trees together inside a shared repository" to "treeless repository with lightweight checkouts". I ran it against a couple of my local repositories and it works out rather nicely. If you are thinking of using cbranch and the like, you should have a look at this plugin. If you don't know why it's a good idea, well, umm, maybe one of the other people on the Planet will blog an answer.

The second adds 'bzr new' to the command list.
$ bzr new awesomer

will create a new shared repository called 'awesomer' and a new branch in that repository called 'trunk'. If you think trunk is a terrible name, you can do:
$ bzr new awesomer devel

If you are like Tim or Aaron, you'll want your repository and branches separated from your working tree:
$ bzr --repository ~/repos/foo foo devel

I'd like to figure out a nice way of letting users specify a default directory for repositories to go into, for those people who always use the split model.

The plugin with both of these commands can be found at https://code.edge.launchpad.net/~jml/+junk/bzr-establish. As the "junk" in the name indicates, it's really rough code.

If either of these features sound like good ideas to you, let me know!

testresources: some examples

There's been a clamor for more documentation on how to use testresources. While I'm not qualified to show you the best way to use it, I can show you the way that I would begin to use it.

The linked files are a bunch of unit tests for database code that uses Storm. Where a real test suite might use an in-memory database, this test suite uses on on-disk sqlite database to better illustrate testresources. This database needs to be removed and built again after each test to guarantee test isolation.

You'll need testresources, Storm and testtools in your Python import path to run these examples.

The first file, complex-example.py, shows how I might do this without testresources. I create a DatabaseService class that has a setUp and tearDown of its own, and a get_store() method that tests are likely to use. This might not be the best thing for databases, but is very close to what I'd do for network services in tests, where I would need to start the service, stop it and have methods to get URLs, clients and other information about the service. If you run the file, you'll see that the database is created at the start of each test, and destroyed before the test finishes. In total, it's created and destroyed three times.

The second file, complex-example-2.py, shows how to switch to using testresources. To do this I,

• added a TestResource subclass called DatabaseResource that implements make() and clean() by delegating to a DatabaseService.
• changed the test case to subclass ResourcedTestCase
• added 'resources = [('database', DatabaseResource())] as a class variable of the test case.

Instead of using DatabaseService() in the test, I could have, and perhaps should have, declared a module level instance of DatabaseResource. In that case, the code would have looked like:

class _DatabaseResource(TestResource):
   ...
DatabaseResource = _DatabaseResource()

class TestPerson(unittest.TestPerson):
  ...
  resources = [('database', DatabaseResource)]
  ...

Running the file shows that the behaviour is the same as the first example: the database is created and destroyed in each test.

The third file, complex-example-3.py, shows how to take advantage of OptimisingTestSuite (sic). We load the test suite as usual, adsorb (sic) them into an OptimisingTestSuite and return that. We also have to explicity declare when a test dirties the DatabaseResource. I chose to do this by adding a dirtied() method to the DatabaseService. If I had used a singleton (as above), then I would have just called dirtied on that.

Anyway, if you run the third example, you'll see that the database is created and destroyed outside the tests and that its only done twice. The test suite has been optimized by sharing resources between tests when possible.

I hope this helps explain how to use testresources. Certainly writing has been a useful exercise for me, it's highlighted that:

• a list of 2-tuples isn't quite right for declaring which resources a test uses.
• the dirtied API is inconvenient
• TestResource remains a confusing name for the class, as it conflates the resource acquisition and cleanup with the actual resource.
• Some of the API docs are wrong (notably the ones for ResourcedTestCase).
• The pattern of "resource object with setUp, tearDown, dirtied (and addCleanup)" should perhaps be turned into Python code.

I long for your thoughts.

Examples:

complex-example.py
complex-example-2.py
complex-example-3.py

pyunit3k renamed to testtools

After thinking and talking about it for ages, I've renamed 'pyunit3k' to 'testtools'. You can now find it at https://launchpad.net/testtools or get your copy by running 'bzr branch lp:testtools'.

This is the only major API change I plan to do without introducing some sort of formal release process. The code as it stands is highly stable, well tested and in use on production systems. Download it, use it, send me patches.

Addendum
I forgot to mention that ITestResult has been removed, and thus the undocumented dependency on zope.interfaces. If this matters to you, please let me know.

LCA 2008 Conference Report

Yeah, yeah, I know it's a bit late. I found this while emptying out my Drafts folder and thought you guys might like it.

It's very much a personal, in-the-moment, opinionated account. I've only edited it for formatting.

Sunday, January 27

Arrived in Melbourne. Good. Next step, dump bags and get beer. Bump into skinny guys with puns on their T-shirts and hunted expressions. Follow them to pub. Rusty is buying free drinks for first-timers. I take two.

Go to Lygon Street. Eat pizza.

Monday, January 28

Public holiday today. No coffee for sale on campus. The buzz so far is that everyone loves Ubuntu, or at least takes it for granted — which can sometimes pass for love.

The chatter about Bazaar is more interesting. Everyone says "Bee-Zed-Ahh" and lists it along with Git and Mercurial. When I talk to people, they don't seem to really care about VCS, they just fall in whatever their project uses. Those who have used DVCS talk about "modern version control" (distributed) and "legacy version control" (CVS, SVN). Those who haven't aren't even aware of how much DVCS can help:

jml: [[ something about branches ]]
other_guy: yeah, I know what branches are.
jml: but we make a new branch for every single bug fix.
other_guy: what? how can you do that? isn't that an pain in the arse?
* jml takes other_guy aside and quietly explains.

The guy who talked about securing code — making sure the stuff you release is the stuff you wrote — knew about Bazaar's GPG revision signing, but didn't like it. I reckon there's some nice stuff we could do here.

At the Debian miniconf, I sat down and listened to someone explain how to use Git to make Debian packages. The speaker kept saying "... because Git is stupid" and then showed us a diagram that had more lines than a telephone exchange.

He also did an interactive demo. Before then, I hadn't seen git being used. It's not a friendly application. I mentioned this to some people on the way out and got answers that were too complex for a hallway. Bazaar rocks, we just need to get people to use it.

Went to the OpenSSH talk. It was packed, and the content was interesting, strong and a little dry. Got some ideas for improving Conch (the SSH server that Launchpad uses for codehosting).

Tuesday, January 29

At the Gnome mini conf, trying to get a leg up into contributing. My grand aim is to get a calendar on my desktop that is as beautiful and pleasurable as iCal.

It seems that lots of others share my desire. I ended up chatting to Rob Bradford from Opened Hand and a couple of other Gnome guys — it sounds like it might be approaching possible.

Found my phone charger.

Wednesday, January 30

Bruce Schneier keynoted. The talk had a few ideas but was light on point. The tutorial on hardening Linux apps was cancelled, and I'm now sitting here while Russell Coker talks about SELinux.

Thursday, January 31

Tired and cranky. Decide to do some hacking.

Friday, February 1

It's 6:45pm now and all I can think about is signing off and zoning out. Lots of cool talks today.

During the Speaker's Panel, someone asked "What was your favourite thing that happened in Open Source last year?". One person put up their hand and said "git".

testresources overhaul

Rob has landed my overhaul of testresources. testresources is now a lot cleaner, with more tests, more documentation and fewer classmethods. If you've looked at it before and found it a bit confusing, now is a great time to have a fresh look.

The NEWS file, README or TODO are all great places to start. Or, you could just download the branch from Launchpad. 'bzr branch lp:testresources' should do the trick.

There's still so much I want to do, but this has been the big step: getting a firm conceptual base to work from.