Relax zfs_vnops_read_chunk_size limitations

It makes no sense to limit read size below the block size, since
DMU will any way consume resources for the whole block, while the
current zfs_vnops_read_chunk_size is only 1MB, which is smaller
that maximum block size of 16MB.  Plus in case of misaligned
Uncached I/O the buffer may get evicted between the chunks,
requiring repeating I/Os.

On 64-bit platforms increase zfs_vnops_read_chunk_size to 32MB.
It allows to less depend on speculative prefetcher if application
requests specific size, first not waiting for prefetcher to start
and later not prefetching more than needed.

Also while there, we don't need to align reads to the chunk size,
but only to a block size, which is smaller and so more forgiving.

My profiles show ~4% of CPU time saving when reading 16MB blocks.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed by: Igor Kozhukhov <[email protected]>
Signed-off-by:	Alexander Motin <[email protected]>
Sponsored by:	iXsystems, Inc.
Closes #17415

Author: amotin

man/man4/zfs.4

@@ -1980,7 +1980,7 @@ Disable QAT hardware acceleration for AES-GCM encryption.
 May be unset after the ZFS modules have been loaded to initialize the QAT
 hardware as long as support is compiled in and the QAT driver is present.
 .
-.It Sy zfs_vnops_read_chunk_size Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq u64
+.It Sy zfs_vnops_read_chunk_size Ns = Ns Sy 33554432 Ns B Po 32 MiB Pc Pq u64
 Bytes to read per chunk.
 .
 .It Sy zfs_read_history Ns = Ns Sy 0 Pq uint

module/zfs/zfs_vnops.c

@@ -99,7 +99,11 @@ static int zfs_dio_strict = 0;
 /*
  * Maximum bytes to read per chunk in zfs_read().
  */
+#ifdef _ILP32
 static uint64_t zfs_vnops_read_chunk_size = 1024 * 1024;
+#else
+static uint64_t zfs_vnops_read_chunk_size = DMU_MAX_ACCESS / 2;
+#endif
 
 int
 zfs_fsync(znode_t *zp, int syncflag, cred_t *cr)
@@ -401,7 +405,8 @@ zfs_read(struct znode *zp, zfs_uio_t *uio, int ioflag, cred_t *cr)
 #if defined(__linux__)
 	ssize_t start_offset = zfs_uio_offset(uio);
 #endif
-	ssize_t chunk_size = zfs_vnops_read_chunk_size;
+	uint_t blksz = zp->z_blksz;
+	ssize_t chunk_size;
 	ssize_t n = MIN(zfs_uio_resid(uio), zp->z_size - zfs_uio_offset(uio));
 	ssize_t start_resid = n;
 	ssize_t dio_remaining_resid = 0;
@@ -432,11 +437,14 @@ zfs_read(struct znode *zp, zfs_uio_t *uio, int ioflag, cred_t *cr)
 		if (dio_remaining_resid != 0)
 			n -= dio_remaining_resid;
 		dflags |= DMU_DIRECTIO;
+	} else {
+		chunk_size = MIN(MAX(zfs_vnops_read_chunk_size, blksz),
+		    DMU_MAX_ACCESS / 2);
 	}
 
 	while (n > 0) {
 		ssize_t nbytes = MIN(n, chunk_size -
-		    P2PHASE(zfs_uio_offset(uio), chunk_size));
+		    P2PHASE(zfs_uio_offset(uio), blksz));
 #ifdef UIO_NOCOPY
 		if (zfs_uio_segflg(uio) == UIO_NOCOPY)
 			error = mappedread_sf(zp, nbytes, uio);