Expand ZFS in Linux VM
Ubuntu 16.04 has native support for ZFS, which means that VMs may start to use ZFS for non-root filesystems. Here’s a cookbook for expanding those filesystems. In OpenStack, the ZFS filesystem must be exported before this can be done, but at AWS it can be done without downtime.
Expanding ZFS in an AWS EC2 instance requires no service interruption.
- Add an EBS volume to the target VM. It will show up as, e.g.,
/dev/xvdf(if the root volume is
Install ZFS libraries and utilities.
# or use aptitude, but it's not always available apt-get install zfs
Create a zpool and zfs filesystem.
# these utilities get installed with the zfs packages. zpool create -o autoexpand=on zp0 /dev/xvdf zfs create -o mountpoint=/data zp0/data
Worth noting at this point is that the ZFS on Linux stuff deals with whole-disk virtual volume by initially creating two partitions: partition number 1 contains the filesystem, partition number 9 is a buffering partition.
Time passes. Work goes on until the filesystem needs expansion.
- Expand the EBS volume, using the AWS web GUI or the command-line tools. This can take a while.
Expand the ZFS partition, first removing the buffering partition.
# after this command, type ‘Fix’ at prompt to take advantage # of new space. parted -l # remove the buffering partition parted /dev/xvdf rm 9 # expand partition holding zpool parted /dev/xvdf resizepart 1 100%
Expand the zpool (and, by extension, the filesystem it contains).
zpool online -e zp0 /dev/xvdf
At that point, the buffering partition is gone. After the next expansion, the resizing can be done without removing any partitions.
Unlike the completely live procedure at Amazon, the OpenStack procedure requires the ZFS filesystem be exported and the virtual drive detached from the VM before the expansion can happen.
- Add a cinder volume to the target VM. It will show up as, e.g.,
/dev/vdb(if the root volume is
Install zfs virtual package.
# use aptitude if available. :-) apt-get install zfs
Create a zpool and zfs filesystem:
zpool create -o autoexpand=on zp0 /dev/vdb zfs create -o mountpoint=/data zp0/data
See AWS note about ZFS creation of two partitions, 1 and 9.
Work goes on until expansion is necessary. Then,
Export the filesystem.
# this is done from within the VM zpool export zp0
Detach the cinder volume from the VM, grow it, and reattach it.
# this is done from a machine with openstack CLI tools # # detach volume openstack server remove volume $VM_GUID $VOL_GUID # resize: new size specified in GB openstack volume set $VOL_GUID --size 16 # attach volume openstack server add volume $VM_GUID $VOL_GUID
Resize the ZFS partition
# this is done from within the VM # # after this command, type ‘Fix’ at prompt to take advantage # of new space. parted -l # remove the buffering partition parted /dev/vdb rm 9 # expand partition holding zpool parted /dev/vdb resizepart 1 100%
Import and expand the zpool
# on import, it will show the pre-expansion size zpool import zp0 # using "online -e" will expand it. zpool online -e zp0 /dev/vdb
Subsequent expansions won’t have to worry about the buffer partition, which is now gone.