Big Data Automation in the Cloud with JobServer
With solutions such as Amazon EMR and Google BigQuery growing in adoption and becoming economically advantageous, business now more than ever need to automate the flow of data between their enterprise storage systems and the cloud. Moving data and information between corporate intranets and the cloud is vital for efficient cloud based Big Data processing.
JobServer's point and click automation and scheduling tools are ideal for centrally managing the flow a data between your Big Data cloud systems such as Amazon EMR and Google BigQuery. JobServer can manage the flow of data to orchestrate the loading and retrieval of data between your Big Data processing systems in the cloud while tracking all your Big Data job processing jobs to give you one place to see everything that is happening in your Hadoop or BiqQuery analytics processing.
In a typical deployment, JobServer sits on your corporate intranet and can load and move data between your in-house storage systems into the cloud for efficient processing then track all Big Data job processing activity to return the necessary critical data and results back in-house or to move it around in the cloud (for example, move data into and out of S3...etc). Alternatively, JobServer can also be easily deployed on the Amazon EC2 or Google GPE instances and run in the cloud. There are multiple topologies possible based how your business operations.
JobServer comes with a built-in and open source plugin API that makes it easy to script and customize Amazon or Google web services apis and create custom tasks and jobs using Java, web services, GWT and python/ruby/bash scripts. For example, you can create complex map reduce jobs in JobServer and get notified when processing is completed and be alerted of any issues at every stage of processing. JobServer lets you also schedule and track detailed realtime and historical reports on all job processing activities whether you are running a Hadoop job, loading a table into the cloud, pulling data back out of BigQuery temp tales, or tracking the progress of BiqQuery query processing itself.
JobServer gives you central control over any automation task you want to perform in the cloud or between activities happening in the cloud and your local enterprise storage and database systems. Try JobServer today and see how you will wonder how you operated with out it.
About Grand Logic
Grand Logic is dedicated to delivering software solutions to its customers that help them automate their business and manage their processes. Grand Logic delivers automation software and specializes in mobile and web products and solutions that streamline business.
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)