Migrating a ZFS RAIDZ2 pool to multiple mirror pools
I’ve had a NAS with ~43.5TB of storage for a while now. However, in the 2-3 years I’ve had it I only used 1.77TB of it. Since I don’t intend to use the space for anything else soon it is best to split the 6 drive pool in 3 pools of 2 drives each. These pools can then be stored at different locations, which is always a good idea in case a disaster happens (e.g. a building catches fire).
Making a backup
Before messing with the current pool, mirror all of it to a spare drive so that if something goes wrong in the process, the data can still be quickly recovered.
# Creates a new pool. Generally you should use /dev/disk/by-id/*, but since # this is temporary it doesn't matter much. zpool create -O compression=lz4 tank_backup /dev/sdX # If you have another pool you can use instead of a spare drive, it is also # possible to create a new datatset zfs create tank/tank_backup
Then, send the data from the main pool to the current pool
# Run this on the receiving machine. (only do this on LAN! the data is not # encrypted). nc -l -p 47424 | zfs recv -v tank/tank_backup # Run this on the sending machine zfs snapshot -r tank@migrate zfs send -Rvc tank@migrate | nc 192.168.2.20 47424 # If you already sent a previous snapshot, you can use this instead zfs send -Rvc -i tank@previous tank@migrate | nc 192.168.2.20 47424
Create a new 2 disk mirror pool
If you are certain the data has been correctly mirrored, you can bring the old pool offline.
zpool export tank
Then, create a new pool:
zpool create -f -O compression=lz4 tank_a mirror /dev/diskid/DISK-a /dev/diskid/DISK-b
Bring the old pool back online
zpool import tank
If you run
zpool status now, you should get something like this:
pool: tank_a state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank_a ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 diskid/DISK-aaaaaaaa ONLINE 0 0 0 diskid/DISK-bbbbbbbb ONLINE 0 0 0 errors: No known data errors pool: tank state: DEGRADED status: One or more devices could not be used because the label is missing or invalid. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the device using 'zpool replace'. see: http://illumos.org/msg/ZFS-8000-4J scan: resilvered 476K in 0 days 00:00:01 with 0 errors on Sat May 1 00:04:20 2021 config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 raidz2-0 DEGRADED 0 0 0 7098730037166419494 FAULTED 0 0 0 was /dev/diskid/DISK-aaaaaaaa 9705479084576219043 FAULTED 0 0 0 was /dev/diskid/DISK-bbbbbbbb diskid/DISK-cccccccc ONLINE 0 0 0 diskid/DISK-dddddddd ONLINE 0 0 0 diskid/DISK-eeeeeeee ONLINE 0 0 0 diskid/DISK-ffffffff ONLINE 0 0 0 errors: No known data errors
Mirror the data
You can send the data from the old pool to the new pool now.
zfs send -Rcv tank@migrate | zfs recv -F tank_a
When that is done, check if the data has been correctly mirrored. Finally, destroy the old pool, create the new pools and mirror the data to both.
zpool destroy tank zpool create -O compression=lz4 tank_b mirror /dev/diskid/DISK-c /dev/diskid/DISK-d zpool create -O compression=lz4 tank_c mirror /dev/diskid/DISK-e /dev/diskid/DISK-f tmux new-session -ds mirror -n 1 'zfs send -Rcv tank_a | zfs recv -F tank_b' tmux new-window -dt mirror -n 2 'zfs send -Rcv tank_a | zfs recv -F tank_c' # May be useful if some of the mountpoints are shared (e.g. /home) zfs set canmount=off tank_b zfs set canmount=off tank_c