NoSQL Zone is brought to you in partnership with:

Aleksa Vukotic is a keen lean and agile advocate, author, trainer, software architect and developer, with experience of leading roles in the successful deliver of business critical projects. Turning to technology, Aleksa has been involve with the Spring Framework since early days, becoming an expert in enterprise Java development with Spring, along with other open source technologies. In addition to his Java EE expertise, Aleksa is often utilising his problem solving skills to tackle the most complex problems arising within the projects. More recently, the focus of Aleksa's technology interest has been the application of NoSQL storage solutions for solving complex data problems. Aleksa is Senior Consultant and Data Management Practice lead at Open Credo, based in London, UK. Aleksa is a DZone MVB and is not an employee of DZone and has posted 5 posts at DZone. You can read more from them at their website. View Full User Profile

Starting with Lift and CouchDB: Resolving Some CouchDB-Record Inconsistencies

12.21.2011
| 2289 views |
  • submit to reddit
In recent months I have been introduced to Lift web framework by my scala-enthusiastic colleagues @ OpenCredo. The scala expressiveness and conciseness along with view-first web approach looked very interesting from the start, especially since great part of my previous experience comes from verbose Java frameworks with traditional MVC architecture.

After initial interest and research I did my first project, using a relational database and the well-known Lift-Mapper framework. Much of the data we needed to store represented documents, so I started experimenting with CouchDB. While the REST interface to Documents and Views in Couch does not require a rich mapping layer (as opposed to the richness  of SQL ORM frameworks), I still found it cumbersome to work directly with HTTP all the time in order to store/query my model objects to/from CouchDB.

One of the nice features of Lift's Record framework is it's agnostic approach to the underlying persistence mechanism, which means that it is relatively easy to plug in any storage solution. Currently the Lift-Record framework has a Record implementation for relational databases (Squeryl), as well as  Mongo-Record and CouchDB-Record implementations which gained prominence with recent increased interest in NoSQL storage. While the CouchDB-Record implementation is not complete, it has enough features to make it useful when working with CouchDB and Lift.  Using CouchDB-Record I could easily map my domain model to the underlying database, and abstract the HTTP layer when I wanted to. The CouchDB-Record implementation did not offer everything that you might expect from a typical O(R)M framework - specifically there are some missing features such as associations mapping, lazy loading, caching. However these features weren't required for this specific project, and some would argue that those are not even a features that a non-relational storage should use (there is no R in ORM if using CouchDB).

First thing to do when using the CouchDB with lift is to add database configuration to the Lift's Boot class. To do this, you need only three lines of code, as Listing 1 illustrates

Listing 1:
val couch = new Database("127.0.0.1",5984, "lift-test")
couch.createIfNotCreated(new Http())
CouchDB.defaultDatabase = couch
To map a model object to CouchDB persistence, you'll need to extend CouchRecord class which is part of the couchdb-record framework. In addition, you define its companion object with the same name, which mixes CouchMetaRecord trait.
Listing 2 shows the sample User class mapped to the CouchDB database using Lift's Record framework

Listing 2:
import net.liftweb.record.field.{DateTimeField, PasswordField, StringField}
import net.liftweb.couchdb.CouchRecord
import com.opencredo.nicheplatform.couchdb.CouchMetaRecord
class User extends CouchRecord[User] {
  def meta = User

  object username extends StringField(this, 20)
  object fullName extends StringField(this, 100)
  object birthDate extends DateTimeField(this)
}
object User extends User with CouchMetaRecord[User] {
}
We demonstrated 2 different field types here: StringField and DateTimeField. For a full list of supported filed types in couchdb-record implementation, you can check the online documentation at http://exploring.liftweb.net/master/index-8.html.
Ok, let's persist a user now. To check that the user is actually peristed we're going to write a test, as illustrated in Listing 3:

Listing 3:
@RunWith(classOf[JUnitRunner])
class UserTest extends FunSuite with LazyLoggable {
  // reset database and boot lift
  if (!Boot.initialized_?) {
    try{
      Http(new Database("127.0.0.1", 5984, "lift-test").delete)
    }catch {
      case e : Exception => Console.print(e)
    }
    val boot: Boot = new Boot()
    boot.boot
  }

 test("Create a new User") {
   val userRecord = User.createRecord #1
   userRecord.username.set("mrbob") #2
   userRecord.fullName.set("Bob Bobic") #3
   val user: Box[User] = userRecord.save #4
   assert(user.isDefined, user.compoundFailMsg("Failure")) #5
 }
}
To create a persistable couchdb-backed document, we simple call the createRecord fuction on our user object (#1). After we set the properties we require (#2, #3), we simply save the created object (#4).

The save function persists the object (by issuing a POST request to the CouchDb server) and returns a Box. The Box will be full if the operation succeeded, in which case the Box will contain the peristed object - with assigned id and rev fields from the CouchDb server.

If the POST operation fails to create the new document the box will be empty, and will contain the Failure message (#5)

The code looks nice, so we run the test, excited about our first couch db persisted object - but the test FAILS miserably!!! What could be wrong.
The test fails at the assertion point (so at least we did the configuration right), and the message is as follows:
Failure(User not defined,Empty,Full(Failure(ok not present in reply or not true: JObject(List(JField(ok,JBool(true)), JField(id,JString(706a63a5ae0adc3325347c8a1d020160)), JField(rev,JString(1-8eaacb2ca383a965037251676b25dbd6)))),Empty,Empty)))
What does this mean? "ok not present in reply or not true", but the JSON object referenced in the rest of the message clearly has the OK message (JField(ok,JBool(true))?
I found this confusing.

What was even more confusing was that the object was actually saved to CouchDB successfully, as I could find it using the Futon web interface.
The fact that the object was actually persisted, and the error message suggested that there was a bug in response parsing code within Lift. I wasn't that familiar with the Lift code base, so after numerous tries,

I found the thread that suggested someone else was having a similar problem (http://groups.google.com/group/liftweb/browse_thread/thread/15d17b9348d4d88e#)
What was the problem? Unfortunately, CouchDB Record implementation didn't follow the work done on other parts of lift framework. The culprit is the following code in CouchDB Record implementation (net.liftweb.couchdb.Database.scala file):

Listing 4:
private[couchdb] object DatabaseHelpers {
  /** Handles the JSON result of an update action by parsing out id and rev, updating the given original object with the values and returning it */
  def handleUpdateResult(original: JObject)(json: JValue): Box[JObject] =
    for {
      obj <- Full(json).asA[JObject] ?~ ("update result is not a JObject: " + json)
      ok  <- Full(json \ "ok" ).asA[JField].map(_.value).asA[JBool].filter(_.value) ?~ ("ok not present in reply or not true: "+json)
      id  <- Full(json \ "id" ).asA[JField].map(_.value).asA[JString].map(_.s)    ?~ ("id not present or not a string: " + json)
      rev <- Full(json \ "rev").asA[JField].map(_.value).asA[JString].map(_.s)    ?~ ("rev not present or not a string: " + json)
    } yield updateIdAndRev(original, id, rev)
}
The argument json: JValue used to be an instance of JField, which the code above correctly maps to concrete JBool/JString instance in order to check the response of CouchDB update operation.

However, some changes to Json parsing since liftweb 2.2 improved this, so that \ operator on the JValue object returns the correct JBool/JString objects directly. This method is called whenever the document is created/updated in CouchDB.

The correct code should look like this:

Listing 5:
private[couchdb] object DatabaseHelpers {
  /** Handles the JSON result of an update action by parsing out id and rev, updating the given original object with the values and returning it */
  /** This is a fix as the original code still depends on old JSON library -- Aleksa**/
  def handleUpdateResult(original: JObject)(json: JValue): Box[JObject] =
    for {
      obj <- Full(json).asA[JObject] ?~ ("update result is not a JObject: " + json)
      ok  <- Full(json \ "ok" ).asA[JBool].filter(_.value) ?~ ("ok not present in reply or not true: "+json)
      id  <- Full(json \ "id" ).asA[JString].map(_.s)    ?~ ("id not present or not a string: " + json)
      rev <- Full(json \ "rev").asA[JString].map(_.s)    ?~ ("rev not present or not a string: " + json)
    } yield updateIdAndRev(original, id, rev)
}
Unfortunately, this change hasn't propagated to CouchDB Record code, and at the time of the writing, there is an open ticket for this bug (http://www.assembla.com/spaces/liftweb/tickets/961-saving-to-couchdb-doesn-t-fetch-the-document--id--property)
Since DatabaseHelpers is the private scala object, it cannot be extended - so you have 2 options:

1. Checkout liftweb, fix the source, build it and use the custom built artefacts in your project. Of course, you'll lose ability to follow future releases of liftweb until the bug is fixed - but hopefully that won't be for too long

2. Override the methods that depend on it, by injecting the correct implementation.

You can easily overwrite the post method on the Database class, which is used by CouchRecord, which all of your CouchDB-mapped objects extend:

Listing 6:
val couch = new Database("127.0.0.1",5984, "lift-test") {
      override def post(doc: JObject): Handler[Box[JObject]] = {
        (JSONRequest(this) <<# doc) ># handleUpdateResultCorrected(doc) _
      }
      def handleUpdateResultCorrecred(original: JObject)(json: JValue): Box[JObject] = {
        //bug-free implementation, as above
      }
}
couch.createIfNotCreated(new Http())
CouchDB.defaultDatabase = couch
This solves the problem of storing new documents to CouchDB - however updates are not handled by this overridden method.
Updates are defined in the Document trait buried within the Database.scala, and this trait is then extend and used for composition quite a few times - not easy to follow, let alone to figure out the clean way to overrirde its behaviour.
You could argue that it is not a significant problem for updates, as you know the document's id and revision values up front, you do not need to confirm them from the response when it succeeds. However, you still have incorrect behaviour, and that is something I cannot live with :)
Rather then spending too much time figuring out how to cleanly override the post method in Document trait, I went down the hard route, checked out the code and fixed the bug, and from there on I used my own custom build lift-couchdb_2.9.0-2.4M3-Aleksa.jar

Now that we have basic tests passing, let's see how we can load the persisted object from CouchDB using its id.
We will continue working on the same test:

Listing 7:
test("Create a new User") {
   val userRecord = User.createRecord
   userRecord.username.set("mrbob")
   userRecord.fullName.set("Bob Bobic")
   val user: Box[User] = userRecord.save

   assert(user.isDefined, user.compoundFailMsg("User not defined"))

   expect("mrbob"){
      user.open_!.username.get
   }
   assert(!user.open_!.id.get.get.isEmpty) #1
   assert(!user.open_!.rev.get.get.isEmpty) #2

   val loadedUser: Box[User] = User.fetch(user.open_!.id.get.get) #3
   assert(loadedUser.isDefined, loadedUser.compoundFailMsg("User not defined"))
   expect("mrbob"){
      loadedUser.open_!.username.get
   }
   expect("Bob Bobic"){
      loadedUser.open_!.fullName.get
   }
   expect(user.open_!.rev.get){
      loadedUser.open_!.rev.get
   }
}
First, we check that the ide and revision fields are now populated as expected (#1, #2).

And the we load the user using the existing id using the fetch method (#3) - it's that simple!

The assertions that follow prove that we loaded all user properties, and that we loaded the document with the latest (expected) revision.
NOTE: Because the CouchRecord's id field is optional, the get function returns the Option object, so we need another get to get to the actual id.

You can use the save function to update the existing record as well. The document to be updated must have id and rev fields set.
Listing 8:
val u2: User = loadedUser.open_!
u2.username.set("bobby")
val updatedUser: Box[User] = u2.save
expect("bobby") {
  loadedUser.open_!.username.get
}
expect(loadedUser.open_!.id.get) {
  updatedUser.open_!.id.get
}
And that's it. Even with few challenges from the Lift's CouchDb implementation, the persistence and fetching of Scala objects to/from CouchDB worked nicelly, using Lift's Record framework.

I will be talking about my experiences on Lift/Scala-CouchDB integration at the NoSQL Exchange at SkillsMatter in London on November 2nd 2011.

Source:  http://www.aleksavukotic.com/2011/09/starting-with-lift-and-couchdb.html

Published at DZone with permission of Aleksa Vukotic, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Robert Craft replied on Thu, 2012/01/26 - 5:12am

´m in touch with CouchDB and the idea of mapping its results to Scala objects, as well as find some natural way to iteract with it, came immediatly. But I see that Dynamic languages such as Ruby and Javascript do things very well with the json/document-centric/shchema-free aproach of CouchDB. Any good aproach to do things with Couch in static languages?

Spring Framework

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.