1. Academy
  2. Volume Store

Create a multi-attach volume with cluster aware file-system with OpenStack CLI

In this tutorial you will learn how to setup a volume as multi attachable within OpenStack.

In this tutorial you will learn how to setup a volume as multi attachable within OpenStack. With this volume it is then possible to use it on multiple instances to share the data.

Prerequisites:

Step 1: Create a security group

For detailed information about setting up a security group see our tutorial How to use security groups.
With the following command we create a new security group:

$ openstack security group create o2cb-access
$ openstack security group show o2cb-access \
--column name
+-------+-------------+
| Field | Value |
+-------+-------------+
| name | o2cb-access |
+-------+-------------+

Then we have to add a rule to allow 7777 for each node that has this security group:

$ openstack security group rule create o2cb-access \
--protocol tcp \
--dst-port 7777 \
--remote-group o2cb-access

 

Step 2: Create the instance / add the security group

The hostname your instance has is important in the next steps of this tutorial!

First we create two instances, if you have already two instances you can skip this command.

The following commands create two Ubuntu 20.04 LTS images using a c3.medium flavor with internet connectivity.

Create the first instance:

$ openstack server create multiv1 \
--key-name <your ssh key name> \
--flavor c3.medium \
--image 'Ubuntu 20.04 LTS' \
--security-group o2cb-access \
--security-group default \
--network public

Create the second instance:

$ openstack server create multiv2 \
--key-name <your ssh key name> \
--flavor c3.medium \
--image 'Ubuntu 20.04 LTS' \
--security-group o2cb-access \
--security-group default \
--network public

You should now have two active instances:

$ openstack server list \
--name 'multiv*' \
--column Name \
--column Status \
--column Image \
--column Flavor
+--------------+--------+------------------+-----------+
| Name | Status | Image | Flavor |
+--------------+--------+------------------+-----------+
| multiv2 | ACTIVE | Ubuntu 20.04 LTS | c3.medium |
| multiv1 | ACTIVE | Ubuntu 20.04 LTS | c3.medium |
+--------------+--------+------------------+-----------+

Step 3: Create the multi attach volume

A multi-attach volume is like a regular volume which can be attached to multiple instances.

Create a multi-attach volume with a size of 10G:
$ openstack volume create shared-volume \
--type multiattach \
--size 10

You should now have a 10G volume of type multiattach:

$ openstack volume show shared-volume \
--column name \
--column status \
--column size \
--column type \
--column multiattach
+-------------+---------------+
| Field | Value |
+-------------+---------------+
| multiattach | True |
| name | shared-volume |
| size | 10 |
| status | available |
| type | multiattach |
+-------------+---------------+

 

Step 4: Attach the volume to the instances

Attach the volume to the first instance:

$ openstack server add volume multiv1 shared-volume \
--os-compute-api-version 2.60

Attach the volume to the second instance:

$ openstack server add volume multiv2 shared-volume \
--os-compute-api-version 2.60

The volume should be automatically recognised by Ubuntu as a new block device.

 

Complete the following step on both instances.

The volume will be made available at /dev/vdX, check the kernel log:

$ egrep 'vd[b-z]' /var/log/kern.log
[ 234.215109] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
[ 234.228468] vdb: vdb1

Step 5: Setup Oracle Cluster File System

For two operating system to write to the same block device you will need a shared-disk file system. Our example will use the Oracle Cluster File System but there are other available options, see Cluster File System on wikipedia.org.

 

Complete the following steps on both instances.

Install Oracle Cluster File System tooling and required kernel modules:

$ sudo apt update
$ sudo apt install --yes ocfs2-tools linux-modules-extra-$(uname -r)
    Enable o2cb in /etc/default/o2cb:
    O2CB_ENABLED=true

    In the next step we are going to create the configuration file for the o2cb setup. Replace the ip_address with the correct ones that are assigned to your instance.

    Create ocfs2 configuration on both instances:

    The name refers to the hostname of the machine, this has to be correct to functionate.

    % sudo tee /etc/ocfs2/cluster.conf <<EOF
    cluster:
    name = ocfs2
    heartbeat_mode = local
    node_count = 2

    node:
    number = 0
    name = multiv1
    ip_address = <ip of instance multiv1>
    ip_port = 7777
    cluster = ocfs2

    node:
    number = 1
    name = multiv2
    ip_address = <ip of instance multiv2>
    ip_port = 7777
    cluster = ocfs2

    EOF
    Restart the o2cb service:
    $ sudo systemctl restart o2cb

    Step 6: Initialize the file system

    Complete the following step on a single instance.

    Validate the correct block device and create an OCFS file-system:

    $ egrep 'vd[b-z]' /var/log/kern.log
    [ 234.215109] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
    [ 234.228468] vdb: vdb1

    $ sudo mkfs.ocfs2 /dev/vdb
    Complete the following step on both instances.

    Mount the new file-system and create a new file appending the hostname of the instance:
    $ sudo mount.ocfs2 /dev/vdb /mnt
    $ hostname | sudo tee --append /mnt/hosts

    The resulting file should contain the names of both instances:

    $ cat /mnt/hosts
    multiv1
    multiv2

    Conclusion

    In this tutorial, you’ve learned how to setup a multi-attach volume through the OpenStack Command Line Interface. Also you made it possible to let it function between two instances.

    More tutorials about using the Command Line Interface can be found here.