Cloud Zone is brought to you in partnership with:

Passionate about technology and startups. Have worked for the BBC in London, Livedoor.com in Japan, Cloudera in San Francisco and MailChannels in Vancouver, Canada. Currently reside in Vancouver where I'm working on building PaaS based on Cloud Foundry, called Stackato. I enjoy writing about technology, especially when it relates to interesting startups. Phil is a DZone MVB and is not an employee of DZone and has posted 50 posts at DZone. You can read more from them at their website. View Full User Profile

Cloud Foundry: Node.js Service Brokers Open-Sourced

05.01.2014
| 4708 views |
  • submit to reddit

[This article was originally written by Jamie Paton.]

We're excited to have open sourced a Cloud Foundry Service Broker implementation in Node.js.

It's been an interesting journey watching the different components of Cloud Foundry evolve since its first release back in 2011. The Data Services are just one of the major components to receive a fresh overhaul in v2 of the Cloud Foundry project.

Why Node.js?

Here at ActiveState we think that having a wide choice of data services allows for greater innovation and optimization of your apps and adding new ones to Cloud Foundry should be a simple process.

We also enjoy using Node.js and taking advantage of the vigorous growth in the community supporting it. With 160+ new modules being added to NPM every day, there is a lot of good code available that's useful for writing REST servers.

At its core Node.js provides everything you need to get started including:

  • REST server using Restify that supports the services v2 API
  • EventEmitter API for subscribing to provision/binding events
  • Built in custom logging
  • The npm module can be used via 'require' or as a standalone broker using the CLI.
  • Provides service record persistence via a LevelDB backing store

Old CFv1 Services

The v1 service architecture generally worked well, but the separation of the service into two distinct components meant that either of these were potential single-points-of-failure. With the v2 services you can set up as many brokers as you need and these can all interface to the same data service to scale horizontally.

Writing v1 services was a slightly awkward affair as you were generally limited to the Ruby implementation which involved building on top of two large Ruby repositories. If the Cloud Controller was updated, then these needed to be updated to ensure compatibility. You also had to house the service processes in a location that could be reached by both a NATS message bus and the Cloud Controller API. This generally restricted services to the same network as the Cloud Foundry cluster itself.

CFv1 Gateways were designed to advertise any number of services to the Cloud Controller that operated under its domain. Its responsibility was to run an asynchronous REST server that solicited bidirectional communication with the cloud controller to advertise the service catalog. It would also maintain synchronization with the service nodes over the local NATS message bus meaning the gateway and node were tightly coupled.

CFv1 Nodes were primarily responsible for managing specific service plans. It was the the primary orchestrator between the actual data service and the gateway and had to maintain the correct set of services dictated by the gateway. Most of the CFv1 core suite of services stored particular provisions of a service within a local SQLite database to ensure consistency.

Detailed information regarding the legacy v1 services implementation can be found here.

New CFv2 Services

The v2 services API brought about a few welcome changes, such as more detailed plan attributes for a more granular billing API, authenticating all requests via HTTP basic auth (previously this was using shared tokens) and a more visible change to how external services are managed with a Cloud Foundry installation. All catalogued services and plans must now have a unique ID assigned to them, and their legacy v1 "version" and "provider" counterparts are deprecated in v2.

Under the hood, the v2 services API introduced a slightly different mode in which you would implement a service. The concepts of the gateway and node as two separate entities no longer exists. Now, there is only a "broker" which is responsible for:

  • Implementing a REST server to interface with the Cloud Controller
  • Authenticating requests using HTTP basic auth
  • Providing an interface to the data service itself for all provision/unprovision & bind/unbind events
  • Maintaining a catalog of all available services and associated service plans
  • Maintaining a consistent record of the provisioned service instances and bindings to ensure services are persistent between broker restarts

Service MarketPlace

Another welcome change was the introduction of a centralized "MarketPlace" system, which allows the CF admin to install services and make them available on a per-organization basis. To enable the marketplace to further support externalized services, the brokers now only require unidirectional communication. This in turn means that you no longer need to expose the Cloud Controller HTTP port over a public or untrusted network to the service broker.

Registering v2 Services

Unlike v1 services, with v2 services you will need to register the broker with a separate service API call. For example, using the cf client:

$ cf create-service-broker <broker name> <user> <password> http://test-broker.stackato.com/

Broker Implementation

We've kept it minimal to allow greater flexibility with your own broker implementations, with the central idea being that you simply supply it with a JSON configuration file detailing the services and associated plans and then simply listen for the events as the cloud controller initiates them:

broker.js:

var Broker = require('cf-services-connector-nodejs');
var Config = require('./config/increment-service'); // JSON configuration file


var inc = 0;


var broker = new Broker(Config);
broker.start();


broker.on('provision', function (req, next) {
    inc++;
    var reply = { dashboard_url: req.params.service_id };
    next(reply);
});


broker.on('unprovision', function (req, next) {
    next();
});


broker.on('bind', function (req, next) {
    var reply = {};
    reply.credentials = {
        host: '192.xxx.xxx.xxx',
        port:  (1000 + inc)
    };
    next(reply);
});


broker.on('unbind', function (req, next) {
    next();
});


// broker.stop();

So far, so simple. Note that your own broker must subscribe to all the events emitted above.

Broker Configuration

With our Node.js implementation of the service broker, all configuration for a broker is encapsulated in a JSON config file.

Here is the configuration file that the above broker uses.

increment-services.json:


{
    "apiVersion": "2.1.0",
    "authUser": "<changeme>",
    "authPassword": "<changeme>",
    "defaultVersion": "1.0",
    "name": "Increment Service",
    "port": 5006,
    "services": [{
        "id": "47c6bff8-c653-4812-89f8-ee9a9b6c3d55",
        "bindable": true,
        "name": "Increment Service",
        "version": "1.0",
        "description": "Increment Service - increment",
        "metadata": {
            "providerDisplayName": "Increment Service Ltd.",
            "tags": [ "increment", "demo" ]
        },
        "plans": [{
            "name": "default",
            "id": "6b6fdd28-eb91-4eb7-9936-bf82ec1ec2d5",
            "description": "This is the first plan",
            "public": true,
            "free": true
            },
            {
            "name": "secondary",
            "id": "a3d1fb32-ba06-4122-9013-d42ef84d8b75",
            "description": "This is the secondary plan",
            "public": true,
            "free": false
        }]
    }],
    "database": {
        "enabled": true,
        "allowOrphanedBindings": false,
        "databaseFile": "./leveldb",
        "encryptionKey": "!!changeme!!"
    }
}
Hopefully, the config above is fairly self explanatory. Most importantly, don't forget to change "authUser", "authPassword" and if using the built in database, "encryptionKey".

If you set the "allowOrphanedBindings" to true, the broker will return a 500 error if it tries to delete a service instance that has bindings. You may want to automatically delete all bindings yourself when a service instance is deleted so this is set to false by default.

The Cloud Foundry documentation for v2 Services describes other configuration attributes you can use. For instance, you can add a documentation URL, support URL and billing information.

Adding The Broker To Your Cluster

The Node.js broker implementation includes a sample broker service, called echo-service.

If we were to deploy this application on the Stackato sandbox, then we could add this echo broker service as an available service on our cluster using the following command.

$ cf add-service-broker \
--username demouser --password demopassword \
--url http://echo-service-broker.stacka.to echo-service

Open Source

We are delighted to have open sourced our service broker implementation to help you start writing your own services. As you can see from the above example, this is now very simple to implement.

Check it out on Github.

We hope this enables you to start effortlessly writing data services for Stackato 3 and Cloud Foundry v2. If you encounter any issues or want to provide feedback, then please open a Github issue or join the #stackato channel on irc.freenode.net.

- See more at: http://www.activestate.com/blog/2014/04/cloud-foundry-nodejs-service-brokers-open-sourced#sthash.p4rXEhjl.dpuf


Published at DZone with permission of Phil Whelan, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)