Support for FS_ALLOW_IDMAP

[mikroskeem]

Describe the feature would like to see added to OpenZFS

Real support for idmapped mounts (MOUNT_ATTR_IDMAP & FS_ALLOW_IDMAP). Initial bits and pieces to get ZFS accept user_namespace parameters were done in #11712

How will this feature improve OpenZFS?

Example use-case would be mounting a snapshot with uid/gid mapped suitable for external archival programs running under non-root user (right now root user, or slower FUSE-based fuse-idmap has to be used to read all files).

Additional context

[almereyda]

Yes, this will greatly enhance LXD experience, where idmapping host volumes UIDs and GIDs to the container is tedious.

Also portable home-directories with systemd-homed become easier.

In a way we already know a similar behaviour from ZFS, when setting a temporary root (-R) for zpools.

Ref.:

• https://lore.kernel.org/lkml/20210213130042.828076-1-christian.brauner@ubuntu.com/T/

[stgraber]

@brauner started digging into this for us (LXD) a little while ago and just now pushed what he had at the time:
https://gitlab.com/brauner/zfs/-/commit/50d3a8d4a93a85ba00382d04d4ffedea5830862d

Worth noting that:

• This is very far from complete, it was mostly about trying to identify where changes would be needed
• Christian no longer works for Canonical and we don't have someone else taking over this work at this time, so if someone else has some interest in getting this sorted, that'd be great to see!

We've in general found those changes to be reasonably small and easy to test for most filesystems that got ported thus far, but ZFS is quite a large beast and the fact that it's not a traditional Linux filesystem (and is multi-platform) makes it quite a bit harder to figure out what needs changing :)

[behlendorf]

@stgraber @brauner thanks for sharing your initial investigatory work in to adding FS_ALLOC_IDMAP support. My hunch is you're right, #11712 plumbed much of what's needed so I'd expect this should be a pretty modest change. Out of curiosity, are there existing test cases we could use to help verify the eventual implementation?

Related to this there are two other open PRs you may be interested in we're trying to wrap up. Any feedback on these changes would be welcome, particularly the user visible aspects (feature flags, CLI functionality, etc). I'd love for the next major OpenZFS version to offer better underlying support for containers.

#12209 - Add support for renameat(2) flags to support overlayfs.
#12263 - Add Linux namespace delegation support.

@allanjude and @cyphar are working on the above changes and might have an interest in pursuing this work.

[brauner]

@stgraber @brauner thanks for sharing your initial investigatory work in to adding FS_ALLOC_IDMAP support. My hunch is you're right, #11712 plumbed much of what's needed so I'd expect this should be a pretty modest change. Out of curiosity, are there existing test cases we could use to help verify the eventual implementation?

Hey @behlendorf!

There are not just a few test-cases, there's a whole test suite that is part of xfstests upstream. The code is:
https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/src/idmapped-mounts/idmapped-mounts.c

If you're familiar with xfstests it's pretty trivial to run only the idmapped mounts tests:

sudo ./check -g idmapped

Related to this there are two other open PRs you may be interested in we're trying to wrap up. Any feedback on these changes would be welcome, particularly the user visible aspects (feature flags, CLI functionality, etc). I'd love for the next major OpenZFS version to offer better underlying support for containers.

#12209 - Add support for renameat(2) flags to support overlayfs. #12263 - Add Linux namespace delegation support.

@allanjude and @cyphar are working on the above changes and might have an interest in pursuing this work.

Interesting, I'll try and find some time.

[youzhongyang]

I have made progress in supporting idmapped mount over zfs, manual testing of local file system operations (such as create, modify, delete and etc.) was successful. When I tried to test over nfs, it didn't work. Here is what I found:

https://github.com/torvalds/linux/blob/master/fs/nfsd/export.c#L431
"exp_export: export of idmapped mounts not yet supported."

Our use cases require nfs access. @brauner - would you please shed some light on why it is not currently supported? is there any technical challenge to support it? Thanks in advance.

[brauner]

On Tue, Jun 28, 2022 at 07:14:58AM -0700, youzhongyang wrote: I have made progress in supporting idmapped mount over zfs, manual testing of local file system operations (such as create, modify, delete and etc.) was successful. When I tried to test over nfs, it didn't work. Here is what I found: https://github.com/torvalds/linux/blob/master/fs/nfsd/export.c#L431 "exp_export: export of idmapped mounts not yet supported." Our use cases require nfs access. @brauner - would you please shed some light on why it is not currently supported? is there any technical challenge to support it? Thanks in advance.

I would need to take a lot how to make this work with NFS exports. Maybe it's easier than I thought but I just didn't feel comfortable exposing this right away. If you get the initial work merged for zfs I can take a look. Note that xfstests have an idmapped mount test suite that you should be able to run on zfs.

[youzhongyang]

Thanks @brauner. I am working on the PR, will make sure to check on xfstests.