ZFS

Robust File System

Tips

• Use /dev/disk/by-id|by-partuuid|by-uuid/ rather than /dev/sd*. This improve drive recognition when swapping them of bay/port and reduce some human errors.
• Resilvering could take a lot of time...

Status

For pools

zpool list
zpool get "all" poolname

For volumes

zfs list
zfs get "all"

Layout

• https://calomel.org/zfs_raid_speed_capacity.html
• https://klarasystems.com/articles/choosing-the-right-zfs-pool-layout/

Disks

• Min = 2Z+1
• Max ~ 4(Z+1)

a-shift

By bit length: 9-> 512, 12 -> 4096

For performance testing:

root@geroda:~ # cat test_recordsize.sh
#! /bin/sh
file=test.tmp
rm $file 2> /dev/null
ST=`sysctl -n vfs.zfs.txg.timeout` 
ST=`expr $ST + 1`
for size in 1 2 3 4 5 6 7 8 9 11 12 13 15 16 17 23 24 25 31 32 33 63 \
        64 65 127 128 129 254 255 256 257; do
    dd if=/dev/random bs=1k count=$size of=$file 2> /dev/null
    sleep $ST
    alloc=`du -k $file | awk '{print $1}'`
    rm $file
    echo "$size K size -> $alloc K alloc"
done

Create

zfs create -s -V 3T poolname/volumename

• -s: for sparse to accept any size

Replace

• https://dannyda.com/2020/05/16/how-to-replace-dead-physical-disk-from-proxmox-pve-for-zfs-pool-easily/#Replace-dead-physical-disk/Replace-with-larger-disks
• https://www.reddit.com/r/zfs/comments/jayyuk/expanding_the_capacity_of_zfs_pool_drives/

Always Zpool offline BAD disk. If not offline, this could disrupt access to the degraded pool.

zpool offline <pool> <device>

In all situation:

zpool replace <pool> <old device> <new device>

Expand

Before or after replacement of all disks

zpool set autoexpand=on <pool>
zpool online -e <pool> <device>

After, good idea to revert

zpool set autoexpand=off <pool>

Compress

zfs set compression=on poolname

ZFS over iSCSI

• https://linuxhint.com/share-zfs-volumes-via-iscsi/

To create the server interface

apt install tgt
tgtadm --lld iscsi --op new --mode target --tid 1 --targetname iqn.YEAR-MM.reversedns:id
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 0 --backing-store /dev/pool/volume
tgtadm --lld iscsi --mode logicalunit --op update --tid 1 --lun 1 --params thin_provisioning=1
tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-address ALL