Formal changes: tests, documentation, copyright

Add zstd to the test suite - add zstd to history_002_pos.ksh - add random levels of zstd to history_002_pos.ksh - add zstd-fast to history_002_pos.ksh - add random levels of zstd-fast to history_002_pos.ksh Add documentation - add man page content - add README for contrib/zstd Fixup copyright headers of touched and new files - Adds copyright headers for Allan Jude / Klara Inc. - Added Ornias1993 Copyright - Cleans copyright header formatting Co-authored-by: Allan Jude <allanjude@freebsd.org> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Allan Jude <allanjude@freebsd.org> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>
openzfs · Dec 17, 2019 · bead964 · bead964
1 parent 9c34044
commit bead964
Show file tree

Hide file tree

Showing 23 changed files with 249 additions and 118 deletions.
diff --git a/contrib/zstd/README.md b/contrib/zstd/README.md
@@ -0,0 +1,20 @@
+
+# ZSTD Contrib Library Manual
+
+
+## Introduction
+
+ZSTD on ZFS uses a heavily cut down, but otherwise unmodified version of ZSTD.
+This `contrib` contains the  unmodified ZSTD library, please do not edit these files.
+
+## Updating ZSTD
+
+To update ZSTD the following steps need to be taken:
+
+1.  Download the latest archive of ZSTD
+2. Replace (not merge): `common`, `compress`, `decompress` and `zstd.h` with the new version from the `lib` folder in the downloaded archive of zstd.
+3. Make sure any newly requered files and/or folders are also included.
+ A. Make sure new folders are listed in: `configure.ac`
+ B. Make sure new Files And Folder are listed in: `lib/libzstd/Makefile.am` and `module/zstd/Makefile.in`
+ C. Make sure any relevant File and Folders are listed in this Readme
+4.  Make sure to change `ZFS_MODULE_VERSION("1.4.4")` in `module/zstd/zstd.c`
diff --git a/include/sys/arc.h b/include/sys/arc.h
@@ -18,10 +18,14 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2012, 2016 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2012, 2016, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef	_SYS_ARC_H

diff --git a/include/sys/spa.h b/include/sys/spa.h
@@ -18,16 +18,20 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2019 by Delphix. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc.  All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
- * Copyright 2013 Saso Kiselkov. All rights reserved.
- * Copyright (c) 2014 Integros [integros.com]
- * Copyright 2017 Joyent, Inc.
- * Copyright (c) 2017, 2019, Datto Inc. All rights reserved.
- * Copyright (c) 2017, Intel Corporation.
+ * Copyright (c) 2011, 2019, Delphix. All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2014, Integros. All rights reserved.
+ * Copyright (c) 2017, Joyent, Inc. All rights reserved.
+ * Copyright (c) 2017, Datto Inc. All rights reserved.
+ * Copyright (c) 2017, Intel Corporation. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _SYS_SPA_H

diff --git a/include/sys/zio.h b/include/sys/zio.h
@@ -21,11 +21,14 @@
 
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2012, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems, Inc. All rights reserved.
+ * Copyright (c) 2012, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
  * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright 2016 Toomas Soome <tsoome@me.com>
+ * Copyright (c) 2016, Toomas Soome. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _ZIO_H

diff --git a/include/sys/zio_compress.h b/include/sys/zio_compress.h
@@ -20,9 +20,11 @@
  */
 
 /*
- * Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
+ * Copyright (c) 2009, Sun Microsystems Inc. All rights reserved.
+ * Copyright (c) 2015, 2016, Delphix. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
  * Use is subject to license terms.
- * Copyright (c) 2015, 2016 by Delphix. All rights reserved.
  */
 
 #ifndef _SYS_ZIO_COMPRESS_H

diff --git a/man/man5/zpool-features.5 b/man/man5/zpool-features.5
@@ -1,7 +1,4 @@
 '\" te
-.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved.
-.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
-.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
 .\" The contents of this file are subject to the terms of the Common Development
 .\" and Distribution License (the "License").  You may not use this file except
 .\" in compliance with the License. You can obtain a copy of the license at
@@ -14,6 +11,11 @@
 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
 .\" own identifying information:
 .\" Portions Copyright [yyyy] [name of copyright owner]
+.\" Copyright (c) 2012, 2018, Delphix. All rights reserved.
+.\" Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+.\" Copyright (c) 2014, Joyent Inc. All rights reserved.
+.\" Copyright (c) 2019, Klara Inc. All rights reserved.
+.\" Copyright (c) 2019, Allan Jude. All rights reserved.
 .TH ZPOOL-FEATURES 5 "Jun 8, 2018"
 .SH NAME
 zpool\-features \- ZFS pool feature descriptions
@@ -927,5 +929,37 @@ is used to checkpoint the pool.
 The feature will only return back to being \fBenabled\fR when the pool
 is rewound or the checkpoint has been discarded.
 
+.sp
+.ne 2
+.na
+\fBzstd_compress\fR
+.ad
+.RS 4n
+.TS
+l l .
+GUID	org.freebsd:zstd_compress
+READ\-ONLY COMPATIBLE	no
+DEPENDENCIES	none
+.TE
+
+\fBzstd\fR is a high-performance compression algorithm that features a
+combination of high compression ratios and high speed.
+Compared to \fBgzip\fR, \fBzstd\fR offers slighty better compression at much
+higher speeds.
+Compared to \fBlz4\fR, \fBzstd\fR offers much better compression while being
+only modestly slower.
+Typically, \fBzstd\fR compression speed ranges from 250 - 500
+megabytes/second/thread, and decompression speed is over 1
+gigabyte/second/thread.
+Booting off of \fBzstd\fR-compressed root pools is not yet supported.
+
+When the \fBzstd\fR feature is set to \fBenabled\fR, the administrator can turn
+on \fBzstd\fR compression of any dataset by running
+`zfs set compress=zstd <pool/fs>`.
+This feature becomes \fBactive\fR once a \fBcompress\fR property has been set to
+\fBzstd\fR, and will return to being \fBenabled\fR once all filesystems that
+have ever had their compress property set to \fBzstd\fR are destroyed.
+.RE
+
 .SH "SEE ALSO"
 zpool(8)
diff --git a/man/man8/zfsprops.8 b/man/man8/zfsprops.8
@@ -19,23 +19,22 @@
 .\" CDDL HEADER END
 .\"
 .\"
-.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
-.\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org>
-.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
-.\" Copyright (c) 2011, Pawel Jakub Dawidek <pjd@FreeBSD.org>
-.\" Copyright (c) 2012, Glen Barber <gjb@FreeBSD.org>
-.\" Copyright (c) 2012, Bryan Drewery <bdrewery@FreeBSD.org>
-.\" Copyright (c) 2013, Steven Hartland <smh@FreeBSD.org>
-.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
-.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
-.\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
-.\" Copyright (c) 2014 Integros [integros.com]
-.\" Copyright (c) 2016 Nexenta Systems, Inc. All Rights Reserved.
-.\" Copyright (c) 2014, Xin LI <delphij@FreeBSD.org>
-.\" Copyright (c) 2014-2015, The FreeBSD Foundation, All Rights Reserved.
-.\" Copyright 2019 Richard Laager. All rights reserved.
-.\" Copyright 2018 Nexenta Systems, Inc.
-.\" Copyright 2019 Joyent, Inc.
+.\" Copyright (c) 2009, Sun Microsystems, Inc. All Rights Reserved.
+.\" Copyright (c) 2011, Joshua M. Clulow. All Rights Reserved.
+.\" Copyright (c) 2011, 2019, Delphix. All rights reserved.
+.\" Copyright (c) 2011, Pawel Jakub Dawidek. All rights reserved.
+.\" Copyright (c) 2012, Glen Barber. All rights reserved.
+.\" Copyright (c) 2012, Bryan Drewery. All rights reserved.
+.\" Copyright (c) 2013, Steven Hartland. All rights reserved.
+.\" Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+.\" Copyright (c) 2014, 2019, Joyent, Inc. All rights reserved.
+.\" Copyright (c) 2014, Adam Stevko. All rights reserved.
+.\" Copyright (c) 2014, Integros. All rights reserved.
+.\" Copyright (c) 2014, Xin LI. All rights reserved.
+.\" Copyright (c) 2014-2015, The FreeBSD Foundation. All Rights Reserved.
+.\" Copyright (c) 2016, 2018, Nexenta Systems, Inc. All rights reserved.
+.\" Copyright (c) 2019, Richard Laager. All rights reserved.
+.\" Copyright (c) 2019, Kjeld Schouten-Lebbing. All rights reserved.
 .\"
 .Dd June 30, 2019
 .Dt ZFSPROPS 8
@@ -723,7 +722,8 @@ for more information on these algorithms.
 Changing this property affects only newly-written data.
 .It Xo
 .Sy compression Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy gzip Ns | Ns
-.Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle
+.Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle Ns | Ns Sy zstd Ns | Ns
+.Sy zstd- Ns Em N Ns | Ns Sy zstd-fast- Ns Em N
 .Xc
 Controls the compression algorithm used for this dataset.
 .Pp
@@ -791,6 +791,35 @@ is equivalent to
 .Pc .
 .Pp
 The
+.Sy zstd
+compression using the zstd algorithm ,
+You can specify the
+.Sy zstd
+level by using the value
+.Sy zstd- Ns Em N ,
+where
+.Em N
+is an integer from 1
+.Pq fastest
+to 19
+.Pq best compression ratio .
+.Sy zstd
+You can also specify a negative
+.Sy zstd
+level by using the value
+.Sy zstd- Ns Em N ,
+where
+.Em N
+is an integer from 100
+.Pq fastest
+to 1
+.Pq best compression ratio .
+.Sy zstd
+is equivalent to
+.Sy zstd-3
+.Pc .
+.Pp
+The
 .Sy zle
 compression algorithm compresses runs of zeros.
 .Pp

diff --git a/module/zcommon/zfeature_common.c b/module/zcommon/zfeature_common.c
@@ -20,11 +20,14 @@
  */
 
 /*
- * Copyright (c) 2011, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright (c) 2014, Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2017, Intel Corporation.
+ * Copyright (c) 2011, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2013, Joyent Inc. All rights reserved.
+ * Copyright (c) 2014, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2017, Intel Corporation. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _KERNEL

diff --git a/module/zcommon/zfs_prop.c b/module/zcommon/zfs_prop.c
@@ -18,16 +18,17 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright 2016, Joyent, Inc.
- * Portions Copyright (c) 2019 by Klara Inc.
+ * Copyright (c) 2010, Robert Milkowski. All rights reserved.
+ * Copyright (c) 2011, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2016, Joyent, Inc. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
-/* Portions Copyright 2010 Robert Milkowski */
-
 #include <sys/zio.h>
 #include <sys/spa.h>
 #include <sys/u8_textprep.h>

diff --git a/module/zfs/arc.c b/module/zfs/arc.c
@@ -18,14 +18,15 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2018, Joyent, Inc.
- * Copyright (c) 2011, 2019 by Delphix. All rights reserved.
- * Copyright (c) 2014 by Saso Kiselkov. All rights reserved.
- * Copyright 2017 Nexenta Systems, Inc.  All rights reserved.
+ * Copyright (c) 2018, Joyent Inc. All rights reserved.
+ * Copyright (c) 2011, 2019, Delphix. All rights reserved.
+ * Copyright (c) 2014, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2017, Nexenta Systems Inc. All rights reserved.
  * Copyright (c) 2019, loli10K <ezomori.nozomu@gmail.com>. All rights reserved.
- * Portions Copyright (c) 2019 by Klara Inc.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
  */
 
 /*

diff --git a/module/zfs/dbuf.c b/module/zfs/dbuf.c
@@ -18,12 +18,16 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc.  All rights reserved.
- * Copyright (c) 2012, 2019 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2012, 2019, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #include <sys/zfs_context.h>

diff --git a/module/zfs/dmu.c b/module/zfs/dmu.c
@@ -18,14 +18,18 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2011, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
  * Copyright (c) 2013, Joyent, Inc. All rights reserved.
  * Copyright (c) 2016, Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2015 by Chunwei Chen. All rights reserved.
- * Copyright (c) 2019 Datto Inc.
+ * Copyright (c) 2015, Chunwei Chen. All rights reserved.
+ * Copyright (c) 2019, Datto Inc. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #include <sys/dmu.h>

diff --git a/module/zfs/dmu_objset.c b/module/zfs/dmu_objset.c
@@ -21,19 +21,21 @@
 
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2012, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
- * Copyright (c) 2015, STRATO AG, Inc. All rights reserved.
- * Copyright (c) 2016 Actifio, Inc. All rights reserved.
- * Copyright 2017 Nexenta Systems, Inc.
- * Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
+ * Copyright (c) 2010, Robert Milkowski. All rights reserved.
+ * Copyright (c) 2012, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2013, Joyent Inc. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2015, STRATO AG. All rights reserved.
+ * Copyright (c) 2016, Actifio Inc. All rights reserved.
+ * Copyright (c) 2017, Nexenta Systems Inc. All Rights Reserved.
+ * Copyright (c) 2017, Open-E Inc. All Rights Reserved.
  * Copyright (c) 2018, loli10K <ezomori.nozomu@gmail.com>. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
-/* Portions Copyright 2010 Robert Milkowski */
-
 #include <sys/zfeature.h>
 #include <sys/cred.h>
 #include <sys/zfs_context.h>