Open Source Review

I started this blog post a few weeks ago, with the intention of taking my time on it. I did, and I am, but I plan to wrap it up before I go to sleep so I think it's important to do it right.

The big news of the day is that we got written about by TechCrunch, but that's not the subject of this post, so I'll probably refrain from blogging about it. Suffice to say that Jan, Damien and I are extremely dedicated to CouchDB, and we are working to get it everywhere that data goes.

Art of Community

I've been reading Jono Bacon's Art of Community and while it can be a hard book to read (look at all the community organizing I'm not doing) there's only one way to get started and that's to start.

Note: You should also be reading Saul Alinsky's Rules for Radicals while you read Jono's book. Alinsky's is full of a brashness and heat that reminds you that this commons-based peer production we're all so interested in has been around since long before life went digital.

A Big Mother

I visited Grandma in the home today. She just turned 90 and she's loving life. It was fun to watch Amy talk to her about things and stuff, and Amy's school friend Erin was there too. They are doing a somatic observation project that involves some freaking cool charts (Amy did hers by hand, it's way better than software could do).

This post is a round-up of all the active (and semi-active) open-source projects I'm involved in. I have a tendency to write fun code when I could be improving existing stuff. After a few years of that, you get a long list of projects. I'll list each project and then do a little rant / elevator pitch. Hopefully it's not a total drag to read.

It's taken me so long to write this partially because it's so hard to admit to yourself that you can't do everything. I want to be able to keep creating new projects at the rate I have been, but to do that and keep the current ones up to date is a lot of work.

What I really need with these is help. I know there are community members out there who have an interest in some of these projects. Now that you can see all the stuff that's on my plate, you'll wonder how I could have time to maintain any of these. I do maintain a few of them, some of them have other people helping already.

Review

Jono's book suggests you do an open source performance review once a year. It's been a little more than a year since I got my CouchDB commit-bit, so I figure now is a good time.

If there's one thing I suck at, it's accepting patches and pull requests (but I'm actually getting better, I think). I'm just not the type of guy who structures my time so that I think about projects I'm not actively using at the current moment. I'm not even great at it when I'm in active development. (I've probably got a dozen versions of jquery.couchapp.js floating around.) This is something I seriously need to work on.

If there's one thing I'm good at, it's biting off more than I can chew and doing it anyway. My first patches to CouchDB touched nearly the whole codebase and I was learning Erlang as I typed.

I think this year will stand out as being a bit exceptional in my overall productivity pattern. I plan on traveling a lot less in 2010, I don't plan on having a disaster restoration company dry clean all the rugs again. I'm starting to feel settled in Berkeley and CouchDB: The Definitive Guide is so close to done.

CouchDB

I've been a committer to the CouchDB project since September 2008 and a contributor a bit longer. Couch is a good environment for new contributors because there is a lot of oversight, and Erlang really isn't that hard. The best way to get started is to find things in Jira that look doable and start filing patches. Matthew Hooker is the latest addition to our THANKS file. His patch was easy to apply because it was small and obvious.

We also like big patches, but chances are you'll have to convince the community that certain changes are good. The best thing to do is involve people up front, as you begin development, so that you know how the community will react. Apache CouchDB development is conducted on the developer mailing list.

CouchRest

CouchRest is the original Ruby CouchDB client I wrote for use in Grabb.it's web-spidering stack. We eventually used CouchDB (together with Nutch, Hadoop, and Hpricot) to power a recommendation engine that tells bands which MP3 blogs they should be promoting themselves too, based on the blog content and linking behavior.

So I had to write a Ruby client for CouchDB, so I ported couch.js to Ruby in probably an afternoon. I used RestClient because I'd met Adam at RubyFringe and I agree with the philosophy of the software. The only thing I never really added was error handling. It's still missing.

I think I mismanaged this project a bit, but the fatal flaw was not inattention, but rather, over-doing it. If there's one thing I could take back from CouchRest it's the 4 hours I spent writing a Ruby DSL for view generation. I suppose it reflects the mindset of the times, but that code led to so much scope-creep.

CouchRest has a nagging problem is lack of clarity about which Github fork to use. I wish Github would make it easier to have a pass-the-baton style of integration lead. This works better on small teams but how many people do you really have patching your library, anyway?

It looks like the mailing list has the release procedure under control. Thank you so much it takes a huge weight off. I may still need to hand the CouchRest Github account password out to a few folks but you know who you are.

Sofa

Sofa arose out of my desire to build a pure CouchApp to replace my aging Rails-based blog engine. It got wrapped up in the O'Reilly book but that's a whole other story. Now I'm still using it and I have plans to add attachment links to the Atom feed so that blog posts with attachments will convey properly. (aka podcast support.)

But more important is stabilizing the code base. I've been developing against trunk for a year just trying to keep all the balls in the air, but it's time to package Sofa. We'll need to think in the CouchDB community about application replication and linking.

But before we get that far I just need to establish a branch for the version of Sofa established CouchDB 0.10. I think a branch is better than a tag for this as I expect a lot of people using that version of CouchDB will be submitting patches, once the book comes out.

CouchApp

A CouchApp is just an app that will run on the garden-variety default installation of CouchDB. For instance, if you write CouchApps, they can be shared amongst all Ubuntu users (and anyone else with a CouchDB.) Canonical was among the first to recognize this with Ubuntu One and Quickly

I think CouchDB's ad-hoc sharing model has the potential to give Ubuntu a fundamental head start against the legacy operating systems. Coupled with the easygoing HTML5 programming model, it'll make peer production of dynamic database backed applications a mainstream phenomenon.

No Really, CouchApp

I'd love to use a Bespin CouchApp Editor. It seems easy like I should be able to throw it together in a weekend but when you're young you never appreciate the sheer volumes of free time you have. I can barely imagine the prospect of a free weekend. I have to carve them out. So I think I'm counting on you to write this one.

What matters is portability.

The whole point of CouchApp as a word is to describe portable applications, that is apps that you can deploy to any-old CouchDB. This is important because the scenario is, everyone already has a CouchDB, so all they need is your app. Easy install, runs local and fast. You win.

And there are lots of other people who have been pushing the CouchApp concept uphill.

A critical path for developers is running compact optimized JavaScript in CouchDB's query server, but working with regular files on the filesystem, in your favorite editor.

Now that's what I call CouchApp

I have to give a huge shout out to Benoît Chesneau who maintains a package that does just that. The CouchApp script is a program that synchronizes a directory on your filesystem with a CouchDB design document. Basically, it makes it so you can edit Map Reduce views in your favorite editor, and deploy to any CouchDB instance you control.

Benoît can crank features out in no time and the code is solid. Thanks Benoît, and everyone else who contributs to CouchApp.

CouchApp could use better documentation (although I tried to be pretty complete in the CouchDB book) but most importantly I think the CouchApp project could use a fresh infusion of use-cases and inspiration. There's a lot more pushing files to Couch than meets the eye (fs-event support, larger deployments, etc) and it'd nice to have a centralized library that keeps it simple for everyone.

Relaxville

Relaxville is probably my favorite CouchApp right now. The goal is a CouchDB community space that operates as a shareable CouchApp. The first feature I contributed is a way to share and report on CouchDB test results. It's still in the "almost works" stage but I already have some artwork I need to integrate. Thanks cinnebar for the creative energy.

Relaxville is really a test of a shared-data CouchApp. It's designed to flush out new requirements for the CouchApp ecosystem. Once we have a simple way to install applications, and to share data via replication, we'll be able to do a bunch of cool stuff. But most importantly, users will be able to distribute their apps to friends or publicly.

I'm working on some patches to make this possible, but there's quite a bit here, so I should outline that work in another post.

Toast

Toast is a realtime chat server which I created to showcase CouchDB's _changes API. It works and can handle a lot of simultaneous users. There are some improvements I'd like to make to it. The biggest change I'd like to make it to package it as JavaScript tag, so it can be easily included into any page that has access to a CouchDB. This means I could use it as the comment handling code for Sofa and Relaxville, among other things.

Calendar

Damien created a Calendar application over a weekend. I'll be posting some slides about the development experience. There are some obvious UI fit and finish improvements to be made. The biggest outstanding feature is probably interoperability with other calendar software. Fixing these two points would take the Calendar from a cool demo, to a useful piece of productivity software.

Analytics

I don't have a project here yet, but it looks like low hanging fruit. I think it should be easy to hook JShub up to CouchDB, and use something like Simile to graph the results. You'd be able to use the analytics package any place you need to measure front-end usage, and you have a CouchDB handy. This would be a good starter project if anyone wants to play with CouchApps.

Twitter Client

Once upon a time, I did all my Twittering using the CouchDB Twitter Client. It's since fallen into disrepair (thanks Tweetie!) but it's got good bones, so you should be able to use it as the basis of more advanced Twitter clients / archives, without much trouble. The client could use updates for changes to the Twitter API. It could also be helped with an additional design document capability (the ability to have CouchDB make HTTP requests to 3rd parties), but the security concerns here are gnarly.

Mimeparse.js

When we added the ability for CouchDB to make responses in non-HTML formats, I ported the Mimeparse library to JavaScript for use by list and show functions. This was a straightforward port from Python. Mostly Mimeparse could use ports to additional languages, but it would also be neat to come up with a declarative test suite, so tht all languages can share the same test suite. Maybe that's over the top, but it could really simplify things.

Conclusion

There are a lot of projects I didn't have time to write up. I've come to the realization that I need to focus on some specific tasks right now. I want to become an excellent open source collaborator, but I need your help to do that.