dracut: zfs_force=1 has no effect

[rkitover]

System information

Type	Version/Name
Distribution Name	Gentoo
Distribution Version	unstable profile default/linux/amd64/17.1/desktop/plasma/systemd
Linux Kernel	5.9
Architecture	amd64
ZFS Version	git master
SPL Version	git master

Describe the problem you're observing

When my zfs pool needs to be force-imported because I booted freebsd on it, the kernel commandline option I added zfs_force=1 does not have any effect, and I have to do a zpool import -f from the dracut rescue shell.

Describe how to reproduce the problem

Add zfs_force=1 to GRUB_CMDLINE_LINUX in /etc/default/grub and regen grub config.

Do something that would require force-importing the pool, in my case that is booting FreeBSD on it.

When you boot linux, you will see dracut fail to import the pool and drop you into the emergency shell.

Include any warning/errors/backtraces from the system logs

[nabijaczleweli]

Could you try the top commit off my suseł-but-stronger branch (referenced above)? It worked in my testing, but, y'know

[rkitover]

Hi, thank you for your response.

I am away from my desktop system right now where I use dracut, on my laptop I have NixOS which I believe does not use dracut.

If I don't get back to my desktop in the next few days to try this, I will just install another distribution on the pool on my laptop, where I have FreeBSD as well already.

How's installing Ubuntu on an existing pool these days? I saw they were doing some serious work in that area. Does Ubuntu use dracut? I'll try it if that's the case.

Your patch looks great by the way.

[bghira]

it works here on Ubuntu, though getting dracut to work on Ubuntu in general is a bit of a gamble.

[rkitover]

So I just got back home recently and tried this on my desktop, with freebsd and gentoo, dracut still drops me to the emergency shell. When I was in the emergency shell, I verified that the patch was applied to my initramfs.

I'm happy to do any debugging.

[nabijaczleweli]

Hm, so if you break in the initrd (or fall through to the emergency shell), these should all be positive:

For the generator:

1. do you definitely have /lib/systemd/system-environment-generators/zfs-import-opts.sh (and it's executable)?
2. does systemctl --system show-environment list ZPOOL_IMPORT_OPTS= at all? (should be -f when you set zfs_force=1)?
3. does it appear if you do systemctl daemon-reload?
4. what do you get on stdout if you run it directly – is it just the ZPOOL_IMPORT_OPTS line?
5. if you append something like set > /dev/console (or /dev/kmsg, or whichever tty you get the shell at) to the generator and reload again, do you get any output (if not – what can you do to get it to run)?

And for the services:

1. do the /lib/systemd/system/zfs-import-{cache,scan}.d/import-opts.conf drop-ins exist? (or maybe they're in the initrd, but in the wrong place? it's entirely possible I got the path wrong!)
2. does systemctl cat zfs-import* list both the service(s) and the drop-ins?
3. if you do systemctl status zfs-import-[the import unit that ran].service – does it list the (value of) $ZPOOL_IMPORT_OPTS in the failed cmdline?

I suspect I just got the permissions/paths/overrides wrong somewhere

[rkitover]

The problem seems to be with my version of systemd not reading the output of the environment generator correctly.

When I do a systemctl daemon-reload I see the WARNING: ZFS will force import pools... etc., and if do a . /lib/dracut-zfs-lib.sh at the prompt then the ZPOOL_IMPORT_OPTS=-f is set in my environment, however systemctl show-environment does not show the variable. I tried removing the leading space before -f but that didn't help.

I did notice that if I do a systemctl set-environment ZPOOL_IMPORT_OPTS=-f then this shows up in systemctl show-environment.

[nabijaczleweli]

Environment generators seem to've landed in systemd 233 (2017-03-01) and remained unchanged since, and the versions I see on packages.g.o are 9999(?) and 248.3, both of which are newer than the 241 from buster that I test on.

I assume the warning you get is from the alternate output, since stderr is devnulled (unless it isn't maybe? but it was in my tests); what happens if you add echo "a b=c" just after the shebang? You should get a warning like

May 30 20:44:32 tarta systemd[8346]: Invalid variable assignment "a b=...", ignoring.

in the journal. If you don't – can you try moving it to, like, /etc/systemd/system-environment-generators maybe?

It might be necessary to fall back to a normal service if your systemd really is so terminally broken so as to have completely non-functional environment generators but I somehow doubt this, given that they've not changed (save for the umask they get) since 4 years ago.

[rkitover]

This system is a couple of months old and my version of systemd is 247.2-r4.

I am going to rebuild my initramfs with vi or something and play with this some more.

[rkitover]

Good news, I found the problem.

It seems that /lib/systemd/system-environment-generators is ignored on my system and the warning I saw was from another invocation of that script.

This may be because this version of systemd only looks in /usr/lib I don't know.

But once I copied the zfs-imports.sh file to /etc/systemd/system-environment-generators, reran daemon-reload and show-environment the variable was set.