Skip to content

Create a multi-attach volume with cluster aware file-system with OpenStack CLI

Estimated time to read: 4 minutes

In this tutorial you will learn how to set up a volume as multi attachable within OpenStack.

With this volume, it is then possible to use it on multiple instances to share the data.

Prerequisites:

  • Two Debian or Ubuntu instances
  • OpenStack CLI Installed
  • OpenRC file from your Fuga dashboard
  • SSH Key added with the OpenStack Credentials you are using in this tutorial -> Create or import existing keypair

Step 1: Create a security group

For detailed information about setting up a security group see our tutorial How to use security groups. With the following command we create a new security group: 

$ openstack security group create o2cb-access
$ openstack security group show o2cb-access \
--column name
+-------+-------------+
| Field | Value       |
+-------+-------------+
| name  | o2cb-access |
+-------+-------------+

Then we have to add a rule to allow 7777 for each node that has this security group: 

$ openstack security group rule create o2cb-access \
--protocol tcp \
--dst-port 7777 \
--remote-group o2cb-access

Step 2: Create the instance / add the security group

Warning

The hostname your instance has is important in the next steps of this tutorial!

First we create two instances, if you have already two instances you can skip this command.

The following commands create two Ubuntu 20.04 LTS images using a c3.medium flavor with internet connectivity.

Create the first instance: 

$ openstack server create multiv1 \
--key-name <your ssh key name> \
--flavor c3.medium \
--image 'Ubuntu 20.04 LTS' \
--security-group o2cb-access \
--security-group default \
--network public
Create the second instance: 
$ openstack server create multiv2 \
--key-name <your ssh key name> \
--flavor c3.medium \
--image 'Ubuntu 20.04 LTS' \
--security-group o2cb-access \
--security-group default \
--network public
You should now have two active instances: 
$ openstack server list \
--name 'multiv*' \
--column Name \
--column Status \
--column Image \
--column Flavor
+--------------+--------+------------------+-----------+
| Name         | Status | Image            | Flavor    |
+--------------+--------+------------------+-----------+
| multiv2      | ACTIVE | Ubuntu 20.04 LTS | c3.medium |
| multiv1      | ACTIVE | Ubuntu 20.04 LTS | c3.medium |
+--------------+--------+------------------+-----------+

Step 3: Create the multi attach volume

A multi-attach volume is like a regular volume which can be attached to multiple instances.

Create a multi-attach volume with a size of 10G: 

$ openstack volume create shared-volume \
--type tier-2m \
--size 10
You should now have a 10G volume of type multi attach: 
$ openstack volume show shared-volume \
--column name \
--column status \
--column size \
--column type \
--column multiattach 
+-------------+---------------+
| Field       | Value         |
+-------------+---------------+
| multiattach |          True |
| name        | shared-volume |
| size        | 10            |
| status      | available     |
| type        | tier-2m       |
+-------------+---------------+

Step 4: Attach the volume to the instances

Attach the volume to the first instance: 

$ openstack server add volume multiv1 shared-volume \
--os-compute-api-version 2.60
Attach the volume to the second instance: 
$ openstack server add volume multiv2 shared-volume \
--os-compute-api-version 2.60
The volume should be automatically recognized by Ubuntu as a new block device.

Complete the following step on both instances.

The volume will be made available at /dev/vdX, check the kernel log: 

$ egrep 'vd[b-z]' /var/log/kern.log
[  234.215109] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
[  234.228468]  vdb: vdb1

Step 5: Setup Oracle Cluster File System

For two operating system to write to the same block device you will need a shared-disk file system. Our example will use the Oracle Cluster File System but there are other available options, see Cluster File System on wikipedia.org.

Complete the following steps on both instances.

Install Oracle Cluster File System tooling and required kernel modules: 

$ sudo apt update
$ sudo apt install --yes ocfs2-tools linux-modules-extra-$(uname -r)
Enable o2cb in /etc/default/o2cb: 
O2CB_ENABLED=true
In the next step we are going to create the configuration file for the o2cb setup. Replace the ip_address with the correct ones that are assigned to your instance.

Create ocfs2 configuration on both instances:

Info

The name refers to the hostname of the machine, this has to be correct to functionate.

% sudo tee /etc/ocfs2/cluster.conf <<EOF
cluster:
    name = ocfs2
    heartbeat_mode = local
    node_count = 2

node:
    number = 0
    name = multiv1
    ip_address = <ip of instance multiv1>
    ip_port = 7777
    cluster = ocfs2

node:
    number = 1
    name = multiv2
    ip_address = <ip of instance multiv2>
    ip_port = 7777
    cluster = ocfs2

EOF
Restart the o2cb service: 
$ sudo systemctl restart o2cb

Step 6: Initialize the file system

Complete the following step on a single instance.

Validate the correct block device and create an OCFS file-system: 

$ egrep 'vd[b-z]' /var/log/kern.log
[ 234.215109] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
[ 234.228468] vdb: vdb1

$ sudo mkfs.ocfs2 /dev/vdb
Complete the following step on both instances.

Mount the new file-system and create a new file appending the hostname of the instance: 

$ sudo mount.ocfs2 /dev/vdb /mnt
$ hostname | sudo tee --append /mnt/hosts
The resulting file should contain the names of both instances: 
$ cat /mnt/hosts
multiv1
multiv2

Conclusion

In this tutorial, you’ve learned how to set up a multi-attach volume through the OpenStack Command Line Interface. Also, you made it possible to let it function between two instances.