Notify userspace when a vdev is removed

[ghost]

Motivation and Context

This is needed for zfsd to autoreplace vdevs on FreeBSD.

Description

Call zfs_post_remove() in spa_async_remove() to tell userspace that the vdev is gone.

How Has This Been Tested?

Tested on FreeBSD by setting autoremove=on, starting zfsd, and pulling out a device to confirm a spare replaces it.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[amotin]

FreeBSD had this for almost 5 years till OpenZFS import.

[behlendorf]

Make sense. Can you also update this comment in zfs_agent_post_event() which is now inaccurate to say something like "OpenZFS kernel modules older than 2.0 don't generate an FM_RESOURCE_REMOVED ereport from the vdev_disk layer after a hot unplug. Fortunately we do get an EC_DEV_REMOVE from our disk monitor and it is a suitable proxy so we remap it here for the benefit of the diagnosis engine. Processing multiple FM_RESOURCE_REMOVED events is not harmful.".

        /*
         * On Linux, we don't get the expected FM_RESOURCE_REMOVED ereport
         * from the vdev_disk layer after a hot unplug. Fortunately we do
         * get an EC_DEV_REMOVE from our disk monitor and it is a suitable
         * proxy so we remap it here for the benefit of the diagnosis engine.
         */

From my reading of the code it should be fine for the ZFS Event Daemon used on Linux to receive multiple copies of this event. So I think we just need to update the comment.

As an aside it looks like the only caller of this function was removed years ago with some other changes which is why it wasn't present in OpenZFS.

[ghost]

• Update comment in zfs_agent_post_event