Migrating a ZFS RAIDZ2 pool to multiple mirror pools
2021-05-01
I’ve had a NAS with ~43.5TB of storage for a while now. However, in the 2-3 years I’ve had it I only used 1.77TB of it. Since I don’t intend to use the space for anything else soon it is best to split the 6 drive pool in 3 pools of 2 drives each. These pools can then be stored at different locations, which is always a good idea in case a disaster happens (e.g. a building catches fire).
Making a backup
Before messing with the current pool, mirror all of it to a spare drive so that if something goes wrong in the process, the data can still be quickly recovered.
# Creates a new pool. Generally you should use /dev/disk/by-id/*, but since
# this is temporary it doesn't matter much.
zpool create -O compression=lz4 tank_backup /dev/sdX
# If you have another pool you can use instead of a spare drive, it is also
# possible to create a new datatset
zfs create tank/tank_backup
Then, send the data from the main pool to the current pool
# Run this on the receiving machine. (only do this on LAN! the data is not
# encrypted).
nc -l -p 47424 | zfs recv -v tank/tank_backup
# Run this on the sending machine
zfs snapshot -r tank@migrate
zfs send -Rvc tank@migrate | nc 192.168.2.20 47424
# If you already sent a previous snapshot, you can use this instead
zfs send -Rvc -i tank@previous tank@migrate | nc 192.168.2.20 47424
Create a new 2 disk mirror pool
If you are certain the data has been correctly mirrored, you can bring the old pool offline.
zpool export tank
Then, create a new pool:
zpool create -f -O compression=lz4 tank_a mirror /dev/diskid/DISK-a /dev/diskid/DISK-b
Bring the old pool back online
zpool import tank
If you run zpool status
now, you should get something like this:
pool: tank_a
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
tank_a ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
diskid/DISK-aaaaaaaa ONLINE 0 0 0
diskid/DISK-bbbbbbbb ONLINE 0 0 0
errors: No known data errors
pool: tank
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://illumos.org/msg/ZFS-8000-4J
scan: resilvered 476K in 0 days 00:00:01 with 0 errors on Sat May 1 00:04:20 2021
config:
NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
7098730037166419494 FAULTED 0 0 0 was /dev/diskid/DISK-aaaaaaaa
9705479084576219043 FAULTED 0 0 0 was /dev/diskid/DISK-bbbbbbbb
diskid/DISK-cccccccc ONLINE 0 0 0
diskid/DISK-dddddddd ONLINE 0 0 0
diskid/DISK-eeeeeeee ONLINE 0 0 0
diskid/DISK-ffffffff ONLINE 0 0 0
errors: No known data errors
Mirror the data
You can send the data from the old pool to the new pool now.
zfs send -Rcv tank@migrate | zfs recv -F tank_a
When that is done, check if the data has been correctly mirrored. Finally, destroy the old pool, create the new pools and mirror the data to both.
zpool destroy tank
zpool create -O compression=lz4 tank_b mirror /dev/diskid/DISK-c /dev/diskid/DISK-d
zpool create -O compression=lz4 tank_c mirror /dev/diskid/DISK-e /dev/diskid/DISK-f
tmux new-session -ds mirror -n 1 'zfs send -Rcv tank_a | zfs recv -F tank_b'
tmux new-window -dt mirror -n 2 'zfs send -Rcv tank_a | zfs recv -F tank_c'
# May be useful if some of the mountpoints are shared (e.g. /home)
zfs set canmount=off tank_b
zfs set canmount=off tank_c