# Migrating a ZFS RAIDZ2 pool to multiple mirror pools

<sup>2021-05-01</sup>

I've had a NAS with ~43.5TB of storage for a while now. However, in the
2-3 years I've had it I only used 1.77TB of it. Since I don't intend to
use the space for anything else soon it is best to split the 6 drive pool
in 3 pools of 2 drives each. These pools can then be stored at different
locations, which is always a good idea in case a disaster happens (e.g. a
building catches fire).

## Making a backup

Before messing with the current pool, mirror all of it to a spare drive
so that if something goes wrong in the process, the data can still
be quickly recovered.

	# Creates a new pool. Generally you should use /dev/disk/by-id/*, but since
	# this is temporary it doesn't matter much.
	zpool create -O compression=lz4 tank_backup /dev/sdX

	# If you have another pool you can use instead of a spare drive, it is also
	# possible to create a new datatset
	zfs create tank/tank_backup

Then, send the data from the main pool to the current pool

	# Run this on the receiving machine. (only do this on LAN! the data is not
	# encrypted).
	nc -l -p 47424 | zfs recv -v tank/tank_backup

	# Run this on the sending machine
	zfs snapshot -r tank@migrate
	zfs send -Rvc tank@migrate | nc 192.168.2.20 47424

	# If you already sent a previous snapshot, you can use this instead
	zfs send -Rvc -i tank@previous tank@migrate | nc 192.168.2.20 47424


## Create a new 2 disk mirror pool

If you are certain the data has been correctly mirrored, you can bring the old pool offline.

	zpool export tank

Then, create a new pool:

	zpool create -f -O compression=lz4 tank_a mirror /dev/diskid/DISK-a /dev/diskid/DISK-b

Bring the old pool back online

	zpool import tank

If you run `zpool status` now, you should get something like this:

	  pool: tank_a
	 state: ONLINE
	  scan: none requested
	config:

		NAME                      STATE     READ WRITE CKSUM
		tank_a                    ONLINE       0     0     0
		  mirror-0                ONLINE       0     0     0
		    diskid/DISK-aaaaaaaa  ONLINE       0     0     0
		    diskid/DISK-bbbbbbbb  ONLINE       0     0     0

	errors: No known data errors

	  pool: tank
	 state: DEGRADED
	status: One or more devices could not be used because the label is missing or
		invalid.  Sufficient replicas exist for the pool to continue
		functioning in a degraded state.
	action: Replace the device using 'zpool replace'.
	   see: http://illumos.org/msg/ZFS-8000-4J
	  scan: resilvered 476K in 0 days 00:00:01 with 0 errors on Sat May  1 00:04:20 2021
	config:

		NAME                      STATE     READ WRITE CKSUM
		tank                      DEGRADED     0     0     0
		  raidz2-0                DEGRADED     0     0     0
		    7098730037166419494   FAULTED      0     0     0  was /dev/diskid/DISK-aaaaaaaa
		    9705479084576219043   FAULTED      0     0     0  was /dev/diskid/DISK-bbbbbbbb
		    diskid/DISK-cccccccc  ONLINE       0     0     0
		    diskid/DISK-dddddddd  ONLINE       0     0     0
		    diskid/DISK-eeeeeeee  ONLINE       0     0     0
		    diskid/DISK-ffffffff  ONLINE       0     0     0

	errors: No known data errors

## Mirror the data

You can send the data from the old pool to the new pool now.

	zfs send -Rcv tank@migrate | zfs recv -F tank_a

When that is done, check if the data has been correctly mirrored. Finally,
destroy the old pool, create the new pools and mirror the data to both.

	zpool destroy tank
	zpool create -O compression=lz4 tank_b mirror /dev/diskid/DISK-c /dev/diskid/DISK-d
	zpool create -O compression=lz4 tank_c mirror /dev/diskid/DISK-e /dev/diskid/DISK-f
	tmux new-session -ds mirror -n 1 'zfs send -Rcv tank_a | zfs recv -F tank_b'
	tmux new-window -dt mirror -n 2 'zfs send -Rcv tank_a | zfs recv -F tank_c'

	# May be useful if some of the mountpoints are shared (e.g. /home)
	zfs set canmount=off tank_b
	zfs set canmount=off tank_c