VDC API: Introduction to the API

Introduction
VDC regions
Prerequisites for using the API
Access to the API server
Keys
Python and pip
Cloudmonkey
Upgrading Cloudmonkey from an earlier version
Cloudmonkey and VDC regions
Using Python with the API
Using Python API commands for VDC regions
Using Cloudmonkey with the API
Parameters and values
Asynchronous commands
Managing response information
Appendix 1: The API signature process
Appendix 2: Using the 'expires' parameter in API requests

 

Introduction

The Virtual Data Centre API gives you control over all the components of your own Virtual Data Centre: virtual machines, storage (disk volumes), networks, templates and ISO images. All of the functions available through the web-based VDC Control Centre can be carried out using the API, plus additional functions, and with the possibility for automation of processes.

To use the API requires the generation of a unique hash-based signature for every call, therefore you will need a scripting or programming language to generate and process commands. Any of the common languages can be used (Python, Ruby, .NET, node.js, etc.). In this documentation, you will find code samples in the form of Python scripts, and guidance for using a free open source command line tool called Cloudmonkey, which provides an easier-to-use interface, based on Python.

In the Appendix you will find the signing process described, in case you wish to implement this for yourself.

For the reference specification of the public API commands, see the API Command Reference.

Note: In this document, commands beginning with '$' are to be typed into the command line, and commands beginning '(local) >' are to be typed into Cloudmonkey.

VDC regions

The VDC zones are grouped into three geographical regions: Europe, North America and Asia. These are managed by separate VDC servers.

For API access, you use the same endpoint API server for all of the regions, and add an extra parameter region to switch between the different regions: 'europe', 'usa' and 'asia' (the region name is not case sensitive). In the following, different ways of using this parameter are explained.

Important

The region parameter is not part of the API command language, it is a special parameter only understood by the VDC API server.

 

Prerequisites for using the API

Access to the API server

The URL you need to use for the VDC API server can be found in My Services, in the same location as your API keys; see below.

Keys

Two key strings, 'API' and 'Secret', are necessary to authenticate your access using the API. Each key pair is assigned to a single username.

You can access your keys using the My Services interface. In the VDC 2.0 window, click the Users tab, and click the Edit button for your own username. There is a radio button setting for API Access as Allow or Deny.

API access is set to Deny by default, and it is a good idea to set it to Deny if you are not using API access for any extended period as this will protect against unauthorised access.

If the Allow is set, then your two keys will be displayed, as well as the URL you should use to access the API server.

If the Deny is set, change it to Allow and press Save. Then press Edit again and the new keys will be visible.

To replace your keys, go through the cycle 'Edit-Deny-Save-Edit-Allow-Save-Edit'.

Python and pip

The official Python download site is http://python.org/download. Python is pre-installed in most versions of Linux. The pip tool is useful for installing Python libraries, which includes Cloudmonkey. pip can be installed from this site: http://www.pip-installer.org/en/latest/installing.html.

Python is also a good choice as a scripting tool for the API, and it is used as the basis for the examples contained in this documentation. The API code has been tested in versions 2.6 and 2.7.

Cloudmonkey

Cloudmonkey is a free, open source command line interface tool for that can be used to communicate with the VDC API. It can be used in interactive mode, which is a good way to explore all of the contents of the API.

Cloudmonkey runs in Linux, Mac OS X and Windows. The current version is 5.3.3 (dated 2016-11-20). Cloudmonkey requires Python version 2, ideally version 2.7; it won't run in Python3.

If you have Python and pip already installed, you can install Cloudmonkey in one step as follows:

$ sudo pip install cloudmonkey

Otherwise, use this website to download the source code: https://pypi.python.org/pypi/cloudmonkey/

Notes for Cloudmonkey and Linux

Note for Ubuntu 14.04 and 14.10: there is a known problem with the version of 'pip' installed in Ubuntu which leads to the following error message when you try to run Cloudmonkey: "Import error in cloudmonkey.cloudmonkey : No module named packages". The solution for this is to completely uninstall the pip program:

$ sudo apt-get remove --purge python-pip

And then install pip again by downloading the file get-pip.py:

$ wget https://bootstrap.pypa.io/get-pip.py

and running the command:

$ sudo python get-pip.py

(Detailed instructions can be found at: https://pip.pypa.io/en/latest/installing.html#install-pip .)

Another problem that we have observed with Ubuntu 14.04/14.10 is that Cloudmonkey fails to run with the error 'Import error in cloudmonkey.requester : No module named packages.urllib3.poolmanager'. The solution for this is to remove/install the 'requests' module:

$ sudo pip uninstall requests
$ sudo pip install requests

Notes for Cloudmonkey and MacOS X

An error you may experience using Cloudmonkey in Mac OS X is that a module 'pkg_resources' is missing (see this StackOverflow question). This can be fixed with the following upgrade command:

$ sudo pip install --upgrade setuptools

Upgrading Cloudmonkey from an earlier version

The easiest way to do this is with the pip installer:

$ sudo pip install --upgrade cloudmonkey

Cloudmonkey and VDC regions

A modified version of Cloudmonkey, created by Interoute [https://github.com/Interoute/cloudmonkey-interoute], is available if you need to access the VDC regions 'North America' and 'Asia'. Use this installation command (which requires that you have the git program installed):

$ sudo pip install git+https://github.com/Interoute/cloudmonkey-interoute.git

Or to upgrade (replace) an existing installation:

$ sudo pip install --upgrade git+https://github.com/Interoute/cloudmonkey-interoute.git

You do not need the modified version if you only use the 'Europe' region.

Using Python with the API

A set of Python scripts is published in the Github repo https://github.com/Interoute/VDC-API-examples-Python which shows how to implement basic Virtual Data Centre tasks as API calls. These all make use of the class VDCApiCall which is defined in the file vdc_api_call.py.

For convenience when using the API, you can store your API server address, API and Secret Keys in a file .vdcapi, like this:

{"api_secret":"n3P50xlPa7IfiA1L7s7hWQplkqHdvkG37tS7ljLZb4X4gpCzKdn93Lu-uEFTUV4h9zboWvCXuvlVcQZeEy9-hg",
"api_key":"HHf9osXIUeIkEUzD__7emXOPuDRug6mpTIVnqDaSIK6NjlwDzq7V0QhvQSil_yk74m9HjIQny7Rb0sgeMsD90B",
"api_url": "http://server/path/to/api"}

In Linux/Mac OS X, this file should be located in your home directory (commonly referred to by the shortcut, ~ [tilde or twiddle]).

Using Python API commands for VDC regions

To access the different VDC regions, you simply add the region parameter/value setting to your list of inputs for an API request. For example, modify zone_get_all.py as follows:

request = {'available': 'true', 'region':'usa'}
result = api.listZones(request)

Using Cloudmonkey with the API

Cloudmonkey can be used both for command-line inputs, or in an interactive mode. Type cloudmonkey to start interactive mode.

For convenience, you can store your API server address, API key and Secret key in a file config located in the folder ~/.cloudmonkey. Here is a sample configuration. You need to replace the values for myaccount, apikey, secretkey, and url with your own values:

[core]
profile = local
cache_file = /home/myaccount/.cloudmonkey/cache
log_file = /home/myaccount/.cloudmonkey/log
asyncblock = true
paramcompletion = true
history_file = /home/myaccount/.cloudmonkey/history

[ui]
color = true
prompt = > 
display = json

[local]
apikey = HHf9osXIUeIkEUzD__7emXOPuDRug6mpTIVnqDaSIK6NjlwDzq7V0QhvQSil_yk74m9HjIQny7Rb0sgeMsD90Bw
url = https://myapiserver.com/path/to/api
expires = 600
secretkey = n3P50xlPa7IfiA1L7s7hWQplkqHdvkG37tS7ljLZb4X4gpCzKdn93Lu-uEFTUV4h9zboWvCXuvlVcQZeEy9-hg
timeout = 3600
username =
password = 
region = europe

The region line will only have an effect if you use the Interoute-modified version of Cloudmonkey. The 'username' and 'password' lines are expected by Cloudmonkey though the login mode of API access is not available for VDC.

Note: The form of the config file changed in Cloudmonkey version 5.2, which introduced profiles. This means that you can maintain different API connections (different servers and/or accounts/keys) and switch between them using set profile.

In interactive mode, Cloudmonkey discovers and caches the available API commands when you run sync as the first command. If you do not sync, a pre-built cache is used, which may not be identical with the actual API available. You can type API commands in their standard form (single words with Camel case), or use Cloudmonkey's simplified format which uses lower case, spaced commands which are grouped by type of command. For example, all commands involving requests for lists of objects are entered with 'list' as the first word of the command:

(local) > list apis

(local) > list virtualmachines

(local) > list zones

and so on (instead of using the standard API commands: listApis, listVirtualMachines, listZones). Cloudmonkey will also tell you which parameters are required for each command, and return an error message if a required parameter is missing, or if any parameter value is not acceptable.

Tip

Cloudmonkey has comprehensive command line completion (use the Tab key or 'double-Tab') to fill in commands and show available options.

 

You can set the VDC region interactively while Cloudmonkey is running. For example:

(local) > set region usa

Note that the current region setting is stored in the config file and it is remembered between Cloudmonkey sessions.

Parameters and values

The API server will check for invalid values of parameters that are themselves valid for the specific command. Similarly, Cloudmonkey will do this as a pre-check before sending a request to the server, and it also performs a pre-check for any required parameters missing from a command. However, there is no checking if parameter names are valid. Thus if you mis-type a parameter name, for example:

(local) > list users accountype=2

the parameter will be ignored by the API server and no error will be reported.

Asynchronous commands

Commands that can take a long time to execute, such as creating disk volumes, are designated as asynchronous. In the API Command Reference, this is indicated by '(A)' after the command name. Or, in the output of the API call listApis, asynchronous commands have the key 'isasync' set to 'true'.

Asynchronous commands immediately return a 'job ID' and you will need to send further calls to the API using queryAsyncJobResult with the job ID to check if the job has completed. There are three possible return values for the 'jobstatus' key:

  • 0: the job is still in progress

  • 1: the job has successfully completed. Any successful response values associated with the asynchronous command are now returned, under 'jobresult'

  • 2: the job has failed to complete. The 'jobresultcode' tag will contain the failure reason code and 'jobresult' will contain the failure reason.

The API command listAsyncJobs returns a list of all of the currently active jobs.

In Cloudmonkey, you can choose how asynchronous commands are to be handled. If you set the configuration asyncblock=true then Cloudmonkey will not return the job ID but instead will query the API and wait until the job has completed, and return the final output. In vdc_api_call.py, the method wait_for_job emulates that Cloudmonkey behaviour. This is used in the example scripts, such as vm_deploy.py.

Managing response information

The VDC API is fairly verbose and many commands return more information than you easily digest from the screen. So it is worth knowing some ways to control the display of that information.

If you are using a scripting language, such as Python, then you are at liberty to control exactly what is displayed. The example Python scripts show some simple cases of that.

Cloudmonkey has some useful control options, including the display format, and the filter option. It is easy to create modified tabular output as follows:

(local) > set display table

(local) > list zones filter=id,name

(local) > list virtualmachines filter=id,name,zonename,cpunumber,memory

You can filter with any top level response tag, as listed in the API Command Reference for that command.

You can use shell pipes and command-line tools such as more, grep or awk. For example, to create a table and display only those machines in the Berlin zone:

(local) > list virtualmachines filter=id,name,zonename,cpunumber,memory | grep Berlin

Or, use more to page a long output:

(local) > list virtualmachines | more

Appendix 1: The API signature process

The API requires all calls to be signed to prove that the user is authenticated to execute the command. To do this you will need the API Key and Secret Key for the VDC user account that you wish to access.

For example, consider the following command to show information about a template. Each API request contains a command name, parameter values, and the API Key value:

command=listTemplates&templatefilter=executable&id=ba54ade9-6b12-4cd7-b671-a9c4a5c957d9&apiKey=HHf9osXIUeIkEUzD__7emXOPuDRug6mpTIVnqDaSIK6NjlwDzq7V0QhvQSil_yk74m9HjIQny7Rb0sgeMsD90Bw

A signature must be added to the end of this, according to the following steps.

  1. For each parameter-value pair, URL encode the value so that it can be sent via HTTP GET (spaces encode to '%20', and so on).

  2. Convert the entire command string to lower case and sort the parts alphabetically using the parameter of every parameter-value pair. The result will look like this:

    apikey=hhf9osxiueikeuzd__7emxopudrug6mptivnqdasik6njlwdzq7v0qhvqsil_yk74m9hjiqny7rb0sgemsd90bw&command=listtemplates&id=ba54ade9-6b12-4cd7-b671-a9c4a5c957d9&templatefilter=executable
  3. Pass this sorted string through the HMAC SHA-1 hashing algorithm with the Secret Key. Base64 encode the resulting byte array. The final signature string obtained, for the above command (and example values of the API Key and Secret Key), will be this: tMgNtGxNVGfen9eNV85JeD4x2dg%3D .

  4. Add this string to the command, so the final full URL should look like:

    https://server/path/to/api?apiKey=HHf9osXIUeIkEUzD__7emXOPuDRug6mpTIVnqDaSIK6NjlwDzq7V0QhvQSil_yk74m9HjIQny7Rb0sgeMsD90Bw&command=listTemplates&id=ba54ade9-6b12-4cd7-b671-a9c4a5c957d9&templatefilter=executable&signature=tMgNtGxNVGfen9eNV85JeD4x2dg%3D

The Python program vdc_api_signer.py shows the signature-building process and it may be useful if you are trying to debug problems with signatures. The program will generate an authentication signature and runnable URL, and optionally execute the API call as a web request.

Appendix 2: Using the 'expires' parameter in API requests

'expires' is an optional parameter for API requests which includes a date-time value, after which the VDC API server will reject the request. This is a good precaution to use against 'man-in-the-middle' attacks (while an attacker could not change the content of the request, which would break the signature, they could send any number of unauthorised copies of the same API request). (Note that Cloudmonkey always uses 'expires' in API calls and the config variable 'expires' is used to set the expiration time in seconds, with a default value of 600).

To use this option, you need to include two additional parameters:

signatureVersion=3

and, for example,

expires=2017-08-23T15:46:30+0000

where the latter contains the date-time at which the API request will expire (using ISO 8601 format).

The program vdc_api_signer.py shows how to implement optional use of 'expires' in API requests.

(While obvious, it is worth pointing out that you can get odd results using 'expires' if the system clock of your sending computer is not synchronised with the VDC API server clock.)