Description
System information
| Type | Version/Name |
|---|---|
| Distribution Name | archlinux |
| Distribution Version | 2021.01.01 |
| Linux Kernel | 5.4.63 and 5.10.10 |
| Architecture | x86_64 |
| ZFS Version | 6fffc88 and fd95af8 |
| SPL Version | n/a |
Describe the problem you're observing
I'm seeing a behavior that I didn't notice until I rebooted for a kernel update. I switched between 5.4 to the 5.10 branch and updated to the latest ZFS HEAD: fd95af8
The cache device in my pool was being written to once per second at a single page: 4k. I booted up a VM and recreated this by just creating a new pool with a cache and observing it write once per second as well.
I dug through the commits from the build I was running from early March 2020, to the build I upgraded from at September 2020 and then to today, 2021-01-25, as they pertained to moudle/zfs/arc.c. I ripped out several commits that were parts of larger efforts, but was unable to get the constant writing to stop. From my test VM, the pool is created and then I immediately watch "zpool iostat" and have used other tools like iotop and iostat which confirm there is indeed a write occurring every second.
I'm sure this is undesirable behavior, if not a bug.
Describe how to reproduce the problem
NOTE: the a.raw, etc, are off by one due to the VM having a root volume for the archlinux install, so the raw files map to vdb+.
Formatting '/vol/kvm/user/archlinux-zfs/a.raw', fmt=raw size=107374182400
Formatting '/vol/kvm/user/archlinux-zfs/b.raw', fmt=raw size=107374182400
Formatting '/vol/kvm/user/archlinux-zfs/c.raw', fmt=raw size=107374182400
Formatting '/vol/kvm/user/archlinux-zfs/d.raw', fmt=raw size=107374182400
Formatting '/vol/kvm/user/archlinux-zfs/e.raw', fmt=raw size=107374182400
$ /opt/kvm.pl --kernel=/boot/linux-5.10.10-host+ archlinux
[root@zfs51010test ~]# uname -a
Linux zfs51010test 5.10.10-host+ #213 SMP Sun Jan 24 17:53:15 CST 2021 x86_64 GNU/Linux
[root@zfs5463test ~]# ls -la /usr/local/zfs
lrwxrwxrwx 1 root root 13 Jan 25 12:00 /usr/local/zfs -> zfs-fd95af8d
[root@zfs51010test ~]# zpool create -f vol /dev/vd[bcde] cache /dev/vdf
[ 431.296776] vdb: vdb1 vdb9
[ 431.440321] vdc: vdc1 vdc9
[ 431.585677] vdd: vdd1 vdd9
[ 431.715964] vde: vde1 vde9
[ 431.857756] vdf: vdf1 vdf9
[root@zfs51010test ~]# zpool iostat -v 1 | grep vdf
vdf 0 100G 1 1 38.5K 7.35K
vdf 0 100G 0 0 0 503
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
vdf 0 100G 0 0 0 511
^C
The writes are occurring once every second, indefinitely. I tried this on my older kernel, 5.4.63, and the behavior is identical.
Include any warning/errors/backtraces from the system logs
None. This isn't a crash but instead runtime behavior.