Kaushik Raghupathi is a senior IT Professional and Project Manager working out of India. Over the years he has worked on numerous IT projects with large sized teams. He is personally very fascinated around Learning methodologies in general and specifically around Community Based Learning with a specific focus on Technology Trends. He is currently experimenting the concepts by working with students in this area. Kaushik has posted 2 posts at DZone. View Full User Profile

Understanding Basic Data Storage Options on Google App Engine

07.22.2010
| 6935 views |
  • submit to reddit

The default storage option available on the App Engine today is the Data Store, a proprietary Google database. In this post we will talk about the Google Data Store and some of the basic aspects of storage, features available and the key benefits and challenges in working with the Data Store.

What is the Google Data Store ?

Google Data Store is a scalable data storage solution for the web applicable, primarily optimized for read and query performance. The key aspect of the Data Store is that it is Entity based storage which means all data is stored and retrieved as entities and attributes. Here are the key features of the Data Store:

  • Google Data Store refers to all data in the form of Entities
  • Each entity has one more named properties
  • A property can reference another entity
  • Supports multiple operations in a single transaction
  • It is not a relational database
  • It transparently handles scalability, performance, replication, load balancing while all these aspects are abstracted from the application developer by usage of APIs

 

Creating Entities

Creating entities involves definining a model of the entity as a class in either Python/Java or other App Engine languages and creating the entity of the class in the constructor by calling the put() method of the API in the class.

Here is an example of a Employee entity in Python

  class Employee(db.Model):  
name = db.StringProperty(required=True)
role = db.StringProperty(required=True,
choices=set(["executive", "manager","producer"]))
hire_date = db.DateProperty()
new_hire_training_completed = db.BooleanProperty()
account = db.UserProperty()
Here are the steps to insert the data into the Data Store

 

    e = Employee(name="",    
role="manager",
account=users.get_current_user())
e.hire_date = datetime.datetime.now().date()
e.put()

Entity Groups allow related entities to be grouped together with a parent child relationship, allowing entities to be updated as part of a single transaction.

Querying Data in the Data Store

Querying Entities from the Data Store can be done through two interfaces: the Query interface and the GqlQuery interface

  • The Query interface prepares a query using instance methods. Here is a usage example for the filter() method retrieving filtered data of an entity
    q = Person.all()q.filter("last_name =", "Smith")
    q.filter("height <", 72)q.order("-height")

    results = q.fetch(5)
  • The GqlQuery interface is an SQL like query interface allowing the application to write queries to operate on the entities.
         q = db.GqlQuery("SELECT * FROM Person " +       
    "WHERE last_name = :1 AND height < :2 " +
    "ORDER BY height DESC",
    "Smith", 72)

    results = q.fetch(5)

Indexes

The process of generating and using indexes in the Google Data Store is transparent to the application developer. Here are some of the key aspects of the Indexing process:

  • An App Engine datastore maintains and uses an Index for every query that an application uses.
  • An Index is a table which contains the result for each query in the desired order
  • While the App Engine application can define its indexes in a configuration file the web server automatically changes this file based on execution patterns
  • The indexes are continuously updated as the queries are executed

The above aspects are sufficient to get started with the Google Datastore and start utilizing its key features. For understanding more details on how the Data Store works and some of the more detailed options refer the Google Documentation

From http://www.thetechtrendz.com/2010/07/understanding-basic-data-storage.html

Published at DZone with permission of its author, Kaushik Raghupathi.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)