CouchDB Edge: Security and Validation

Damien committed some new security features just before the holidays. I've got to go through and understand them anyway, so here's a blog of the dive.

h2. First stop, couch_tests.js

Anytime you want to catch up with CouchDB's bleeding edge, the first stop is couch_tests.js. It's just a series of Javascript tests that run from the browser. They're pretty simple to understand, if you've ever used a CouchDB client library in any language. (The Ajax calls execute synchronously, so you don't even have to do the Javascript callback time warp in your head.)

Here what I'm looking for: the security_validation test. And in the middle of it, it's clear that everything is running in an exceptional state, because the body of the test runs as a closure passed to: run_on_modified_server.

Damien's documentation here says a lot:

  // This tests couchdb's security and validation features. This does

  // not test authentication, except to use test authentication code made

  // specifically for this testing. It is a WWWW-Authenticate scheme named

  // X-Couch-Test-Auth, and the user names and passwords are hard coded

  // on the server-side.

Ok, so the upshot is that the tests are against a stub of the real authentication method. The onus is on developers to come up with workable and secure authentication schemes. I've heard talk about LDAP integration. We'll, let's see what run_on_modified_server does.

Simple enough: it just modifies the settings with this new piece of .ini:

    authentication_handler = {couch_httpd, special_test_authentication_handler}

    WWW-Authenticate = X-Couch-Test-Auth

Then, with modified settings, it runs the test closure. Once the closure is complete (including raising errors), it puts CouchDB back the way it was. Ok, so don't run the tests on a public production server, but we already knew that...

h2. CouchDB's authentication handlers

We've got to see the mock, before we can understand the tests. My next move is a "Find in Project" for special_test_authentication_handler, which turns out to have some funny easter eggs from Damien: I think my favorite is @nslater's password: "biggiesmalls endian".

special_test_authentication_handler(Req) ->

    case header_value(Req, "WWW-Authenticate") of

    "X-Couch-Test-Auth " ++ NamePass ->

        % NamePass is a colon separated string: "joe schmoe:a password".

        {ok, [Name, Pass]} = regexp:split(NamePass, ":"),

        case {Name, Pass} of

        {"Jan Lehnardt", "apple"} -> ok;

        {"Christopher Lenz", "dog food"} -> ok;

        {"Noah Slater", "biggiesmalls endian"} -> ok;

        {"Chris Anderson", "mp3"} -> ok;

        {"Damien Katz", "pecan pie"} -> ok;

        {, } ->

            throw({unauthorized, <<"Name or password is incorrect.">>})

        end,

        #user_ctx{name=?l2b(Name)};

    _ ->

        % No X-Couch-Test-Auth credentials sent, give admin access so the

        % previous authentication can be restored after the test

        #user_ctx{roles=[<<"_admin">>]}

    end.

So this function will authorize the commiters to log in with goofy passwords and have our name appear in the #user_ctx{}. This sounds like a reasonable mock. Let's see what the default handler looks like.

  default_authentication_handler(Req) ->

      case basic_username_pw(Req) of

      {User, Pass} ->

          case couch_server:is_admin(User, Pass) of

          true ->

              #user_ctx{name=User, roles=[<<"_admin">>]};

          false ->

              throw({unauthorized, <<"Name or password is incorrect.">>})

          end;

      nil ->

          case couch_server:has_admins() of

          true ->

              #user_ctx{};

          false ->

              % if no admins, then everyone is admin! Yay, admin party!

              #user_ctx{roles=[<<"_admin">>]}

          end

      end.

couch_httpd:default_authentication_handler is not any longer than the mock - let's see how it works. First it parses the requests using basic_username_pw which will return the tuple {User, Pass} if they are present as Basic Auth.

CouchDB uses an htpassword like utility to setup admin accounts. It looks like the default basic auth only checks for admin status. There is no built-in facility for storing user data in a database, or authenticating via an external couchjs server, but this looks like a pretty good place to put one.

If anyone feels moved to write an couch_httpd:external_authentication_handler, I'm sure it would be right at home here. I think encapsulating authentication in a seperate external process (from the one that runs the validations) will make applications more robust in the long run.

couch_db:prep_and_validate_new_edit triggers the update function to run inside the view server. If you want to understand the full stack here, also grep for validate_doc_update and refresh_validate_doc_funs. There's liberal use of Erlang closures (to hold the connection to the view server port) which I always find relaxing.

h2. Writing apps with auth

We've seen so far that after authentication proceeds, the user's Name is sent to the security and validation functions. Now that we have a deeper understanding of the internals and what they can offer us, let's jump back to couch_tests.js to see security and validation in action.

The test creates a design document, with a field called validate_doc_update, which contains a JavaScript function which looks like:

function(newDoc, oldDoc, userCtx) {

  // docs should have an author field.

  if (!newDoc._deleted && !newDoc.author) {

    throw {forbidden:

        "Documents must have an author field"};

  }

  if (oldDoc && oldDoc.author != userCtx.name) {

      throw {unauthorized:

          "You are not the author of this document. You jerk."};

  }

}

The rule stated in this function is that users may not edit the documents unless they created them, and that documents must record an author field.

Potential Feature Opportunity: It'd be nice to be able to modify the doc before saving here (for instance to add a timestamp or the user name), but it could compromise write performance to do a 2-way serialization of the document between CouchDB and the validation server. Maybe if it were optional to complete the function with return docToSave;, people could use it as they saw fit.

In the test we see a few interesting behaviors:

• You can't save a document if you have the wrong password.
• You can save a document if you have the correct password.
• Regular users can't save design docs.
• Admins can add DB admins via the HTTP API.
• Anyone can add the first admin: "if no admins, then everyone is admin! Yay, admin party!"
• Admins can save design documents
• Once the design doc is in place, no on can save docs without an author field.
• Jan can't edit documents whose author is Damien.
• Damien can save a doc under Jan's name, then Jan can edit it.
• Damien can't delete the doc now, but Jan can.
  
  Once our applications have a reliable authentication mechanism, these behaviors should be enough to setup most of the validity scenarios we need. The only missing piece here is the ability to define and use roles other than _admin. If I have a CMS I might like to require that users have a particular role in order to save articles. This should be an easy enough feature to support in an _external validation server.
  
  An interesting conundrum - how to store user account documents in CouchDB so that external processes launched by the Couch can query them, but they are not available for public access?
  
  h2. Take Home Message
  
  CouchDB validation and security features are bare-bones, but on the right track to provide what we'll need as developers to control the way our applications save data to CouchDBs. The programming model is simple right now, but full-featured enough for prototyping systems. Implementing authentication handlers is the sort of thing I expect the open-source community to be good at, so on the whole, I think we're headed in the right direction.
  
  I'm working on porting the software that runs this blog to a pure CouchDB application, but it looks like I may have to dive in and write a more generic auth server first (that handles more than admin status and interfaces well with Ajax apps). I'll keep you posted with that work as it progresses.