NoSQL Zone is brought to you in partnership with:

Doug has been engrossed in programming since his parents first bought him an Apple IIe computer in 4th grade. Throughout his early career, Doug proved his flexibility and ingenuity in crafting solutions in a variety of environments. Doug’s most recent work has been in the telecom industry developing tools to analyze large amounts of network traffic using C++ and Python. Doug loves learning and synthesizing this knowledge into code and blog articles. Doug is a DZone MVB and is not an employee of DZone and has posted 36 posts at DZone. You can read more from them at their website. View Full User Profile

SubRedis: Keyspaces in Redis

01.08.2014
| 4431 views |
  • submit to reddit

We’ve been using Redis extensively for Quepid, our search relevancy collaboration canvas. Redis is a convenient and efficient way to interact with in-memory data structures. In fact, we’ve been using it as our primary database.

One thing that’s stuck out as problematic for us is isolating and segmenting parts of our application within Redis. I want to give each part of Quepid’s backend a safe and reliable Redis sandbox, without running 10 instances of Redis on my box. For example, if the user database does a flushdb, I want it to be only resetting the set of users. Similarly, I don’t want to worry that keys within the user database will clash with keys with the query-ratings database. Isolation is a good thing!

Introducing SubRedis

So what I’ve done is create Subredis — a simple Python keyspace wrapper around Redis. In short, SubRedis lets you do this:

r = redis.from_url("redis://localhost:6379")
sub = SubRedis("keyspacename", r) #My very own Redis!
sub.set("foo", "bar")
sub.get("foo")
sub.flushdb()

Each SubRedis plays entirely in its own keyspace. Here for example, “flushdb” will only ever flush keys in “keyspacename” and the key “foo” will exist only here in this keyspace.

Redis has traditionally been a very flat key-value store. However, SubRedis allows us to create hierarchy within Redis. For example, to get the user’s set of queries for Quepid, we can easily do:

queryRedis = SubRedis(userId + "-queries", redis)

And viola, each user has their own Redis instance ready to go and wholly owned by them. Moreover, there’s no reason a SubRedis cannot in turn aggregate a SubRedis itself, achieving arbitrary levels of hiererachy:

queryRedis = SubRedis(userId + "-queries", redis)
ratingsRedis = SubRedis(queryId + "-ratings", queryRedis)

Pretty powerful stuff!

More Details

Subredis is a simple wrapper, all it does is prepend the name “keyspacename” to keys it receives. So a call in redis-cli to “keys *” before flushdb above would yield:

keyspacename_foo

SubRedis implements most of the methods of redispy’s StrictRedis with a few exceptions where it may not be possible to intercept the keys. Every data structure is supported. Transactions are supported via pipelines. We currently don’t support many of the admin methods for safety sake (ie bgsave etc) and Lua scripting is not feasible with this approach. As SubRedis uses the “_” delimiter, its not encouraged to create key names with “_” in them.

Nevertheless, this has been extremely convenient when developing Quepid. Each component can trust in having its own isolated Redis instance, separated from other components.

So if you’re a Redis user, give SubRedis a try! Simply pip install subredis and let me know what you think (and if you find bugs!). I hope to blog about the fun Python metaprogramming that made this relatively simple to perform in the future.

And if you’re one of those people still wondering what data structure is right for your job, let us know! It may be Redis, or it could be any number of other things we specialize in (Cassandra, Solr, Hadoop, etc).


Published at DZone with permission of Doug Turnbull, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)