Cloud Zone is brought to you in partnership with:

Cloudify is a DZone MVB and is not an employee of DZone and has posted 37 posts at DZone. You can read more from them at their website. View Full User Profile

OpenStack Orchestration with Heat + Devstack

07.08.2014
| 3624 views |
  • submit to reddit

[This article was originally written by Shay Naeh.]

OpenStack Heat + Cloudify + Devstack

Today in order to bootstrap a Cloudify manager you have to take care of the OpenStack compute, networking and security services yourself, essentially all of the OpenStack orchestration (this is still a good reference).  Each of these is comprised of a number of configurations. 

Just to name a few:

  • When we talk about compute services, these can include provisioning the right instance type, memory and CPU characteristics, image type, and more.
  • Networking services include networks, ports, subnets, association with floating IPs as well as an establishment of a router and binding its interfaces correctly.  See this post to learn more about OpenStack Networking.
  • Security services include the creation of security groups to define which network ports are open and whom can access them, as well as the creation of security keys to SSH into the manager machine, and future instances machines. There is a different key for the manager machine and and a different key for the launched application instances.

Below is an example of the bootstrapped Cloudify manager after all the above IaaS services are provisioned.

image

The Heat Stack

Moreover, when creating the Cloudify manager in the usual way you have to install a CLI (command line interface), which in turn connects to Openstack endpoints like the Identity endpoint and Neutron endpoint, and communicates with them over the network using JSON-based commands. This works fine, but since there are more components involved like installing a CLI, it may take more time and is at times more error prone.

To simplify this, you can create an OpenStack Heat stack, and launch the stack which will create everything automatically. Later on I'll show a real example of how to launch the Cloudify manager as a Heat stack.


OpenStack orchestration made easy. Test drive Cloudify.     Go


First let's talk about Heat. Heat is based on a HOT template – (which we first discussed in this post on OpenStack Heat if you need a refresher),  which resembles the TOSCA standard. The template is composed of three main sections:

  • The parameters: Where you can define various parameters and constants to be referenced later when creation and accessing resources.
  • The resources section: Where you define and provision resource like compute resources, network, subnets, etc.
  • The output section: Where you can output variables like the web page URL, public IPs, etc.

Heat stacks can be nested, and one stack can reference a second stack. This is what is called Template Composition with Amazon CloudFormation.

As we shall see next, you can define IaaS services and dependencies between them, for example network creation comes before a port creation or a subnet creation.

Cloudify Manager provisioned as a Heat Stack

heat_template_version: 2013-05-23

description: >
  Cosmo manager template

parameters:
  key_name:
    type: string
    description : Name of a KeyPair to enable SSH access to the instance
    default : shay
  instance_type:
    type: string
    description: Instance type for management server
    default: m1.large
    constraints:
      - allowed_values: [m1.small, m1.medium, m1.large]
        description: instance_type must be one of m1.small, m1.medium or m1.large
  image_id:
    type: string
    description: ID of the image to use for the management server
    default: fc08d66a-31ba-41af-9a07-5bbf4546a7b0 
 
  manager_net_cidr:
    type: string
    description: manager network address (CIDR notation)
    default: 10.10.1.0/24
  manager_net_gateway:
    type: string
    description: manager network gateway address
    default: 10.10.1.1
  manager_net_pool_start:
    type: string
    description: Start of app network IP address allocation pool
    default: 10.10.1.4
  manager_net_pool_end:
    type: string
    description: End of app network IP address allocation pool
    default: 10.10.1.34
  public_net_id:
    type: string
    description: The ID of the public network. You will need to replace it with your DevStack public network ID
    default: 09c20c92-e654-480d-bce4-f0bc1f9fc785 
  

resources:

  manager_network:
      type: OS::Neutron::Net
      properties:
        admin_state_up: true
        name: manager_network
        shared: true
        tenant_id: admin

  manager_subnet:
      type: OS::Neutron::Subnet
      properties:
        network_id: { get_resource: manager_network }
        cidr: { get_param: manager_net_cidr } 
        gateway_ip: { get_param: manager_net_gateway }
        dns_nameservers: [8.8.8.8, 8.8.4.4]
        allocation_pools:
          - start: { get_param: manager_net_pool_start }
            end: { get_param: manager_net_pool_end }

  router:
      type: OS::Neutron::Router

  router_gateway:
      type: OS::Neutron::RouterGateway
      properties:
        router_id: { get_resource: router }
        network_id: { get_param: public_net_id }

  router_interface:
      type: OS::Neutron::RouterInterface
      properties:
        router_id: { get_resource: router }
        subnet_id: { get_resource: manager_subnet }

  manager_instance:
    type: OS::Nova::Server
    properties:
      image: { get_param: image_id }
      flavor: { get_param: instance_type }
      key_name: { get_param: key_name }
      networks:
        - port: { get_resource: manager_port }
      user_data:
        str_replace:
          template: |
            #!/bin/bash -v
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-components_3.0.0-beta-b1_amd64.deb -P /cloudify
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-core_3.0.0-beta-b1_amd64.deb -P /cloudify
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-ui_3.0.0-beta-b1_amd64.deb -P /cloudify-ui
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/ubuntu-agent_3.0.0-beta-b1_amd64.deb -P /cloudify-agents

            sudo dpkg -i /cloudify/*.deb
            sudo /cloudify-components/cloudify-components-bootstrap.sh
            sudo /cloudify-core/cloudify-core-bootstrap.sh ubuntu manager_host  
            sudo dpkg -i /cloudify-ui/*.deb
            sudo dpkg -i /cloudify-agents/*.deb

            #http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/ubuntu-agent_3.0.0-beta-b1_amd64.deb

          params:
            manager_host: {get_attr: [manager_port, fixed_ips, 0, ip_address]}


  manager_port:
      type: OS::Neutron::Port
      properties:
        network_id: { get_resource: manager_network }
        fixed_ips:
          - subnet_id: { get_resource: manager_subnet }
  
  manager_floatingip:
      type: OS::Neutron::FloatingIP
      properties:
        floating_network_id: { get_param: public_net_id }
        port_id: { get_resource: manager_port }
        
outputs:
  WebsiteURL:
    description: URL for manager
    value:
      str_replace:
        template: http://host
        params:
          host: { get_attr: [manager_floatingip, floating_ip_address] }

You can see above a real example of how to install the Cloudify manager as a Heat stack. In the parameters section we define several parameters, like the key to SSH to the manager machine, instance type, image ID, the manager network in CIDR notation, the gateway address, start/end network allocation pool, as well as the public network ID.

In the resources section, we first create the network, then the subnet . You can see how we reference the parameters defined in the previous section like gateway_ip. We also reference the Google name servers as the DNS name servers to resolve names to IP addresses. Next, we provision the router and bind its interfaces. And now after all of the IaaS services are provisioned, and are in place, we can provision the Cloudify manager with the right image_id, flavor, and key_name. In the user data we create a bash script which installs the manager packages. First it downloads the right packages from the net, and then installs them.

In the output section we retrieve and present the manager’s floating IP address.

Below are two charts, the first shows how the heat stack looks graphically, with all of the connected nodes and dependencies.

The second chart shows the provisioned resources:  Networks, subnets, Floating IPs and Cloudify manager VM.

image

image

In order to access the Cloudify manager from an external browser I've created port forwarding on the Devstack machine, so that when you access port 8080 of the Devstack VM hosted on HP Cloud, you will be forwarded to port 80 on the Cloudify manager provisioned VM (this is specific to to this current example, but this can easily be provisioned on any Openstack cloud). The commands to create the port forwarding are listed below.

image

In the chart below you can see a screenshot of how the bootstrapped Cloudify manager looks when it’s up and running.

image

Published at DZone with permission of Cloudify Community, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)