VDC API: How to create and manage storage volumes

Introduction
Types of storage volumes in VDC
Get a list of available storage volume types (diskofferings) (API command: listDiskOfferings)
Create a storage volume (API command: createVolume)
List the storage volumes in a VDC account (API command: listVolumes)
Attach a volume to a virtual machine (API command: attachVolume)
Partitioning and formatting a storage volume
Mounting a storage volume
Detach a volume from a virtual machine (API command: detachVolume)
Resize a storage volume (API command: resizeVolume)
Delete a storage volume (API command: deleteVolume)

Introduction

This document describes how to use the VDC API to create and manage storage volumes, and how to attach volumes to virtual machines.

The API code examples in this document use the Cloudmonkey API tool. For installation and setup, see Introduction to the API.

Types of storage volumes in VDC

When a VM is created a dedicated storage volume of type ROOT is created to hold the VM operating system. It is recommended not to use this 'root disk' to hold application or user data, and the root disk is deployed at a minimal size sufficient to accommodate only the operating system and essential applications software. The root disk can be resized at any time if more space is required; the VM must be stopped during the resizing.

You can add a new storage volume to your VM at any time. These volumes have the type DATADISK. They can be resized, and data protection services are available.

There is a choice of magnetic disk storage (EBS) or solid state storage (SSD) and there are three tiers of data protection:

  • Unprotected, without any backup of the disk contents;

  • Protected, which has automatic backup of the disk inside the same VDC zone;

  • Mirrored, which has dual automatic backup in the same VDC zone and a remote VDC zone (only offered for EBS disks).

Note: Please check the VDC 2.0 Technical Data Sheet for current operational limits on storage disk sizes and performance. Please check the Pricing web page for the latest pricing information.

Get a list of available storage volume types (diskofferings) (API command: listDiskOfferings)

Use this command to get the UUIDs for the different storage types:

(local) > list diskofferings filter=name,displaytext,id
+-----------------+---------------+--------------------------------------+
|   displaytext   |      name     |                  id                  |
+-----------------+---------------+--------------------------------------+
|       EBS       |      EBS      | 71537cd8-3641-43e5-86d2-c7755b5cd924 |
|     EBS SSD     |    EBS SSD    | 89f79c15-d38b-4d69-ad8e-e6c2efd5f4c7 |
|    Protected    |   Protected   | e3752c57-6742-4e45-b259-c6b225f155a7 |
|  Protected SSD  | Protected SSD | 67ec69a7-45de-491d-84b8-321bedf30a21 |
|     Mirrored    |    Mirrored   | 2b421334-29cf-4f90-ae91-c9732ea980e3 |
+-----------------+---------------+--------------------------------------+

'EBS' and 'EBS SSD' are the Unprotected storage options.

These UUIDs are for the Europe region. The UUIDs are different in the Asia and North America regions.

Ignore the 'root disk' offerings which appear in the output (and not shown above). These can only be used for creating root disks in VM deployments based on ISO images for operating systems.

Create a storage volume (API command: createVolume)

Here is the Cloudmonkey API command to create a new 'Protected SSD' datadisk of size 200 GBytes in the Frankfurt VDC zone (use command listZones to get the required UUID):

(local) > create volume name=datadisk-FRA-ubuntu01 size=200 zoneid=7144b207-e97e-4e4a-b15d-64a30711e0e7 diskofferingid=67ec69a7-45de-491d-84b8-321bedf30a21
cmd = org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd
created = 2017-07-18T13:16:29+0000
jobid = dfeb1e7c-f0bc-4fa0-aa73-ed4b9a797743
jobprocstatus = 0
jobresult:
volume:
id = f692c558-6e52-4cd8-b431-e0fa262a6f40
name = datadisk-FRA-ubuntu01
created = 2017-07-18T13:16:29+0000
diskofferingdisplaytext = Protected SSD
diskofferingid = 67ec69a7-45de-491d-84b8-321bedf30a21
diskofferingname = Protected SSD
size = 214748364800
state = Allocated
type = DATADISK
zoneid = 7144b207-e97e-4e4a-b15d-64a30711e0e7
zonename = Frankfurt (ESX)

(Output abbreviated.)

The 'name' input should be meaningful for your own purposes.

Disk 'size' can be any whole number quantity of GBytes between 10 GB and 2000 GB.

List the storage volumes in a VDC account (API command: listVolumes)

This command get the details of all of the volumes in a VDC account. There are many request and response parameters, only a selection are shown here.

For example:

(local) > list volumes filter=id,name,vmname,zonename,deviceid,type
+--------------------------+----------+----------+-----------------------------------+--------------------------------------+-----------------+
|           name           |   type   | deviceid |               vmname              |                  id                  |     zonename    |
+--------------------------+----------+----------+-----------------------------------+--------------------------------------+-----------------+
|  datadisk-FRA-ubuntu01   | DATADISK |          |                                   | f692c558-6e52-4cd8-b431-e0fa262a6f40 | Frankfurt (ESX) |
|       ROOT-401614        |   ROOT   |    0     |      rancher-host-istanbul-1      | beec1035-30ca-4bd4-b2f6-a613d47ede10 |  Istanbul (ESX) |
|       ROOT-366040        |   ROOT   |    0     |       Webcluster-NFS-server       | 2946294b-57df-48f0-b428-067a86dd32d0 | Frankfurt (ESX) |
|       ROOT-401613        |   ROOT   |    0     |      rancher-host-istanbul-2      | 736f6c8d-3bf8-4d50-9953-e665e60377f8 |  Istanbul (ESX) |
| Webcluster-NFS-datadisk  | DATADISK |    1     |       Webcluster-NFS-server       | 3b8af552-1e36-43a4-8c70-b4d78a843a6c | Frankfurt (ESX) |
+--------------------------+----------+----------+-----------------------------------+--------------------------------------+-----------------+

By default you will see both ROOT and DATADISK volumes. If you want to select one of these then use the 'type' parameter:

(local) > list volumes filter=id,name,vmname,zonename,deviceid type=DATADISK

When a datadisk is attached to a VM the response parameters 'deviceid', 'vmname', etc, are returned with values. The response for an unattached datadisk does not have these parameters.

To get the data for one volume only, use the the 'id' parameter with the volume's UUID, for example:

(local) > set display json
(local) > list volumes id=3b8af552-1e36-43a4-8c70-b4d78a843a6c
{
  "count": 1,
  "volume": [
    {
      "account": "Interoute VDC Demo",
      "attached": "2016-08-01T14:36:13+0000",
      "created": "2016-08-01T14:33:36+0000",
      "destroyed": false,
      "deviceid": 1,
      "diskofferingdisplaytext": "Protected SSD",
      "diskofferingid": "67ec69a7-45de-491d-84b8-321bedf30a21",
      "diskofferingname": "Protected SSD",
      "domain": "Interoute VDC Demo",
      "domainid": "5b195d44-b1af-4540-a976-a6e162857e8b",
      "id": "3b8af552-1e36-43a4-8c70-b4d78a843a6c",
      "isextractable": true,
      "name": "Webcluster-NFS-datadisk",
      "provisioningtype": "thin",
      "quiescevm": false,
      "size": 53687091200,
      "state": "Ready",
      "storagetype": "shared",
      "tags": [],
      "type": "DATADISK",
      "virtualmachineid": "f953f2c8-e50c-4f8c-bd49-916fdbea89e7",
      "vmdisplayname": "Webcluster-NFS-server",
      "vmname": "Webcluster-NFS-server",
      "vmstate": "Stopped",
      "zoneid": "7144b207-e97e-4e4a-b15d-64a30711e0e7",
      "zonename": "Frankfurt (ESX)"
    }
  ]
}

Attach a volume to a virtual machine (API command: attachVolume)

You can attach a volume to a VM at any time. It is not necessary (for VDC) to stop a VM to attach a volume, however it is usually necessary to re-start a VM so that the operating system correctly detects the attached disk.

You simply need to specify the volume UUID and and the virtual machine UUID, for example:

(local) > attach volume id=f692c558-6e52-4cd8-b431-e0fa262a6f40 virtualmachineid=096417e2-5eaa-40a3-bf92-f9c92b04ff3c

(You will need to substitute the UUIDs for your own volume and VM.) Attachment is only possible when the volume and VM are in the same VDC zone.

A new datadisk must be partitioned and formatted to be used by the operating system, as in the next section.

Partitioning and formatting a storage volume

After attaching a storage volume, you will need to restart the VM so that the operating system detects the new disk, which in a Linux OS will appear initially as '/dev/sdb' if it is the first attached disk ('sda' being the root disk).

For Ubuntu Linux, the steps to partition and format the disk are as follows :

ubuntu1404:~$ sudo fdisk /dev/sdb

This command presents a series of prompts, for which the simplest case (single partition with default sectors) the values to enter are: 'n', 'p', '1', press enter for the default options for the sectors, then finally 'w' to write this configuration to the disk. A formatted disk device '/dev/sdb1' now exists. (Multiple partitions of a disk volume would appear as sdb1, sdb2, etc.)

For use with a Linux virtual machine, the 'ext4' format is a suitable choice for a data disk:

ubuntu1404:~$ sudo mkfs -t ext4 /dev/sdb1

Mounting a storage volume

A simple disk mount in Ubuntu is as follows:

ubuntu1404:~$ sudo mkdir /media/datadisk01
ubuntu1404:~$ sudo mount /dev/sdb1 /media/datadisk01

The mount point name 'datadisk01' is arbitrary and at your choice. See Ubuntu: Installing a New Hard Drive for more details about the disk installation process in Ubuntu.

All file content stored in '/media/datadisk01' will be stored permanently on the VDC datadisk, which can be detached and stored offline or re-attached to a different virtual machine.

Detach a volume from a virtual machine (API command: detachVolume)

You can detach a volume from a VM at any time. It is not necessary (for VDC) to stop a VM to detach a volume, however it is usually necessary to re-start a VM so that the operating system correctly detects the removal of a disk.

(local) > detach volume id=f692c558-6e52-4cd8-b431-e0fa262a6f40

Resize a storage volume (API command: resizeVolume)

To resize a DATADISK volume it should be unattached, or the VM which it is attached to should be in 'stopped' state.

A ROOT volume can be resized when its VM is 'stopped'.

Important

Resizing to a smaller size is not possible. You will need to copy the old drive data into a new and smaller volume.

For example, to resize a volume to 300 GBytes:

(local) > resize volume id=f692c558-6e52-4cd8-b431-e0fa262a6f40 size=300

Delete a storage volume (API command: deleteVolume)

An unattached datadisk volume can be deleted. A deleted disk is kept for 72 hours (the 'expunge period') and during this time it can be recovered by submitting a request ticket to Interoute Support.

Example:

(local) > delete volume id=f692c558-6e52-4cd8-b431-e0fa262a6f40

Deleting a volume does not remove any snapshots of the volume which have been taken (see How to take a snapshot of a volume).