cannot receive new filesystem stream: checksum mismatch or incomplete stream.

[fantacinni]

Hello,

I've an issue when restoring the pool from a snapshot image file.
The snapshot was sent by
sudo zfs send -Rwv tank@now | zstd -cq -o tank.zst

Tank pool was destroyed after sent and recreated, the recv process was interrupted with the following message.
unzstd tank.zst -c | sudo zfs recv tank -Fsv -o compression=zstd

receiving full stream of tank@send into tank@send
received 42.2K stream in 1 seconds (42.2K/sec)
receiving incremental stream of tank@zrepl_20230216_184621_000 into tank@zrepl_20230216_184621_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230217_191155_000 into tank@zrepl_20230217_191155_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230220_163336_000 into tank@zrepl_20230220_163336_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230222_170034_000 into tank@zrepl_20230222_170034_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230223_173759_000 into tank@zrepl_20230223_173759_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230224_191543_000 into tank@zrepl_20230224_191543_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230226_183237_000 into tank@zrepl_20230226_183237_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230228_142256_000 into tank@zrepl_20230228_142256_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230301_175955_000 into tank@zrepl_20230301_175955_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230302_163347_000 into tank@zrepl_20230302_163347_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230302_173347_000 into tank@zrepl_20230302_173347_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@zrepl_20230303_123216_000 into tank@zrepl_20230303_123216_000
received 312B stream in 1 seconds (312B/sec)
receiving incremental stream of tank@now into tank@now
received 312B stream in 1 seconds (312B/sec)
receiving full stream of tank/storage@send into tank/storage@send
received 18.6G stream in 62 seconds (308M/sec)
receiving incremental stream of tank/storage@zrepl_20230216_184620_000 into tank/storage@zrepl_20230216_184620_000
received 11.2M stream in 1 seconds (11.2M/sec)
receiving incremental stream of tank/storage@zrepl_20230217_191154_000 into tank/storage@zrepl_20230217_191154_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230222_170033_000 into tank/storage@zrepl_20230222_170033_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230223_173758_000 into tank/storage@zrepl_20230223_173758_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230224_191543_000 into tank/storage@zrepl_20230224_191543_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230226_183237_000 into tank/storage@zrepl_20230226_183237_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230228_142256_000 into tank/storage@zrepl_20230228_142256_000
received 111K stream in 1 seconds (111K/sec)
receiving incremental stream of tank/storage@zrepl_20230301_175956_000 into tank/storage@zrepl_20230301_175956_000
received 104K stream in 1 seconds (104K/sec)
receiving incremental stream of tank/storage@zrepl_20230302_170347_000 into tank/storage@zrepl_20230302_170347_000
received 18.8G stream in 62 seconds (311M/sec)
receiving incremental stream of tank/storage@zrepl_20230303_123216_000 into tank/storage@zrepl_20230303_123216_000
received 8.89M stream in 1 seconds (8.89M/sec)
receiving incremental stream of tank/storage@now into tank/storage@now
received 312B stream in 1 seconds (312B/sec)
receiving full stream of tank/storage/Clips@send into tank/storage/Clips@send
cannot receive new filesystem stream: checksum mismatch or incomplete stream.
Partially received snapshot is saved.
A resuming stream can be generated on the sending system by running:
    zfs send -t 1-114f15e9dd-118-789c636064000310a500c4ec50360710e72765a526973030188b42d460c8a7a515a79680650c60f26c48f2499525a9c540bae34fcfc478664cfd25f9e9a599290c0c070d171d7ed3f0cbd500499e132c9f97989b0aa413f3b2f58b4bf28b12d353f59d73320b8a1d8a53f352c0e6f13220dc9f9358949e9a94939f9c9d9f0d129380da03934fcd4d4a4d014a81f471238927e7e71614a516174374410000e8882b43

Since this is the only single copy of the pool, and I really want to restore it.
Any help is appreciated!

Edit: code formatting

[matthiasg]

i have the same error trying to send from one machine to another. how can there be an error like this? is this an error in the source ?

[bill-mcgonigle]

I know this is old, but since it's still open, if you want to try to save the majority of the data you can try:

echo 1 > /sys/module/zfs/parameters/zfs_send_corrupt_data

and run the send/recv.

Bad blocks will be filled in with 0x2f5baddb10c which you can search for to see which files got corrupted. But at least your good files will be restored.