The P2P Web (part one)

(Part one because this is only just some of what I'd like to say on the topic.)

The web was originally designed as a peer-to-peer medium. Tim Berners-Lee needed a way to share physics papers with his friends around the world. Since they were physicists, and the medium was simply published texts, the barriers to entry were low. All you had to do to become an independent publisher was run a copy of the NCSA web server, and point it to an HTML directory of your papers.

To make a long story short, the idea caught on. In the 15 years since we've seen an explosion of uses for the web, alongside an steady increase in the complexity of web applications and deployments. Google's web index is a long way away from the humble list of all the university web servers of the early years. Even much smaller sites (like the ones you might be building right now) are more likely to consist of service-oriented architecture and complex caching protocols, than a directory index of static html files.

This complexity comes for a reason, the web has gotten bigger and users have grown to expect sites to integrate data from all kinds of sources. Even if your site is simple enough to be hosted as plain old html, you're still subject to the traffic spikes that millions of active users can bring without warning.

More importantly, all these changes have made it that much harder for the average user to run their own web sites. At the same time, companies have stepped into the gap to make it easy to publish a blog or connect with your friends (as long as you host your writing or your friendships on the company server.) Most users have never considered controlling their own platforms, instead we see frustration about control manifest itself in ways that many of us geeks find amusing: clamboring to grap "your" Facebook URL before someone else does, pushing the limits of how ugly one can style a Myspace page, anger at site operators for not implementing your favorite imaginary features. These are all reactions to the very real power site operators have over their users.

There are also more-geeky reactions: the OpenID and Data-Portability movements, for instance. While these geek movements have their heart in the right place, I was never able to get excited about them. For as much as they see the value of giving control to users, they're still steeped in the idea of a web owned by vendors, where users can at best "go on strike" to demand more respect. The data-portability movement may have had user's interests at heart, but the ability to send your photos from Flickr to Picasa isn't a huge step up.

I suppose if there's a similar term to describe the p2p web, it'd be "application portability." When an application is designed to run against a client-based CouchDB node, not only can it (and the data it touches) be replicated from (say) Flickr to Picasa, it can also be replicated from Alice to Bob, or even modified by Alice and then replicated for publishing by any generic CouchDB hosting provider.

I tend to dig into the technical details of how this is accomplished, but I'll skip that part for today. Instead I'll just say it's easy. Once you've got a group of people with CouchDB on their laptops, it only takes a few minutes before they are actively sharing and aggregating data in an ad-hoc way. Since applications are just another form of data, they are replicated along with other changes. The big picture here seems to be a little bit harder for experienced developers to grasp, than for new developers and end users.

If you've never jumped through the mental hoops required to build a modern web application cluster, if you've never struggled with defining indexes on a relational database or configuring an HTTP proxy, than you have less invested in the centralized model of application development. I've found that people who haven't yet learned how "to do it right" are quite comfortable with the relaxed model CouchDB provides. "You mean I just save this document and then load it again when I need it?" "I can get this same data onto your computer by clicking that button?"

What is simple for a user requires a lot of relearning on the part of experienced developers. The physics of the web are changing, and a lot of the hard work we had to do to make the centralized model function just doesn't apply to the p2p web. So far I've said more about the centralized web than the peer-to-peer web. I should at least mention that I don't see the centralized web going anywhere anytime soon. There will always be room for online shopping carts and even centralized message routers. But as a general rule of thumb technology has followed the path of least resistance.

When users have the data they care about locally, they can afford to burn much more CPU time pulling interesting patterns from it than (say) Google can use to categorize a particular web page. This in turn frees the centralized services to provide what they do best: message routing and peer discovery. It's nice that Facebook can help me find photos of my friends, but it's frustrating that each time I visit an album I have to wait for image files to cross the wire. Social graph services of the future will be built on the assumption that users have data they may want to share with their friends but not the service provider. It's up to them to figure out how to meet that need.