Increase L2ARC write rate and headroom

Current L2ARC write rate and headroom parameters are very conservative:
l2arc_write_max=8M and l2arc_headroom=2 (ie: a full L2ARC writes at
8 MB/s, scanning 16/32 MB of ARC tail each time; a warming L2ARC runs
at 2x these rates).

These values were selected 15+ years ago based on then-current SSDs
size, performance and endurance. Todays we have multi-TB, fast and
cheap SSDs which can sustain much higher read/write rates.

For this reason, this patch increases l2arc_write_max to 32 MB and
set l2arc_headroom to 0. The former option speed up L2ARC warming and
renewal rate, while the latter instruct the L2ARC write thread to
search into all ARC (rather than on a short tail) for eligible buffers.

With persistent L2ARC, l2arc_headroom=0 is especially valuable as it
effectively enable a sort of "persistent ARC" - if/when the machine
reboots, most of ARC is restored into L2ARC, providing a very tangible
performance boost compared to an otherwise "cold" ARC/L2ARC.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>

module/zfs/arc.c

@@ -776,8 +776,8 @@ uint64_t zfs_crc64_table[256];
  * Level 2 ARC
  */
 
-#define	L2ARC_WRITE_SIZE	(8 * 1024 * 1024)	/* initial write max */
-#define	L2ARC_HEADROOM		2			/* num of writes */
+#define	L2ARC_WRITE_SIZE	(32 * 1024 * 1024)	/* initial write max */
+#define	L2ARC_HEADROOM		0			/* num of writes */
 
 /*
  * If we discover during ARC scan any buffers to be compressed, we boost