cmd/link: fallocate_test.go:46: unexpected disk usage: got 1 blocks, want 2048

[wheelcomplex]

What version of Go are you using (go version)?

$ go version
go version devel +c875503cf7 Sun Jun 28 03:14:10 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

no reproduced in go-1.14.4

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/david/home/.cache/go-build"
GOENV="/home/david/home/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/david/sandspace/pkgextgo/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/david/sandspace/pkgextgo/"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/home/david/sandspace/go"
GOSUMDB="sum.golang.org"
GOTMPDIR="/home/david/sandspace/tmp"
GOTOOLDIR="/home/david/sandspace/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/home/david/sandspace/tmp/go-build907894934=/tmp/go-build -gno-record-gcc-switches"

What did you do?

compile Go from source: cd src && ./all.bash

What did you expect to see?

ALL TESTS PASSED

What did you see instead?

ok  	cmd/link/internal/benchmark	0.086s
--- FAIL: TestFallocate (0.00s)
    fallocate_test.go:46: unexpected disk usage: got 1 blocks, want 2048
    fallocate_test.go:46: unexpected disk usage: got 1 blocks, want 4096
    fallocate_test.go:46: unexpected disk usage: got 1 blocks, want 6144
FAIL
FAIL	cmd/link/internal/ld	5.307s

What did you figured out:

fallocate_test.go is new from:

39ea0ea

@cherrymui @aclements

This issue may involve with ZFS on Linux (zfs, 0.8.3, Linux kernel 5.4.0-39-generic).

The test is passed if run in a directory on filesystem ext4 or btrfs (COW filesystem similar to ZFS) but it is failed on ZFS.

[ianlancetaylor]

CC @jeremyfaller @cherrymui @thanm

[thanm]

Thanks for sending this bug report.

It would be helpful to know how your ZFS storage pool is set up. Could you please share the output of "zfs get all" or "zfs get recordsize"?

[cherrymui]

The number of blocks is reported by stat's st_blocks, which is

blkcnt_t  st_blocks;      /* Number of 512B blocks allocated */

So either fallocate doesn't allocate file size as expected, or stat doesn't report the number of blocks as expected.

Could you try the following C program? Thanks.
(assuming /tmp/xxx does not exist. change to other path if you like)

#define _GNU_SOURCE             /* See feature_test_macros(7) */
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>

int main() {
	int e;
	int fd;
	struct stat st;

	fd = open("/tmp/xxx", O_RDWR|O_CREAT|O_TRUNC, 0666);
	if (fd < 0) {
		printf("open failed: %d\n", fd);
		return 1;
	}
	e = fallocate(fd, 0, 0, 1<<20);
	if (e < 0) {
		printf("fallocate failed: %d\n", e);
		return 1;
	}
	e = fstat(fd, &st);
	if (e < 0) {
		printf("fstat failed: %d\n", e);
		return 1;
	}
	printf("st_size/512 = %d/512 = %d, st_blocks = %d\n", st.st_size, st.st_size/512, st.st_blocks);
	return 0;
}

[wheelcomplex]

for @cherrymui

It is ok when run on btrfs filesystem:

st_size/512 = 1048576/512 = 2048, st_blocks = 2048

it returns 'fallocate failed: -1' on ZFS, if run by strace we got:

munmap(0x7f4efacc0000, 179742)          = 0
openat(AT_FDCWD, "./xxx.blk", O_RDWR|O_CREAT|O_TRUNC, 0666) = 3
fallocate(3, 0, 0, 1048576)             = -1 EOPNOTSUPP (Operation not supported)
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0
brk(NULL)                               = 0x557cd95fc000
brk(0x557cd961d000)                     = 0x557cd961d000
write(1, "fallocate failed: -1\n", 21fallocate failed: -1
)  = 21
exit_group(1)                           = ?
+++ exited with 1 +++

There is an issue address this: openzfs/zfs#326

@thanm

$ zfs get all tank
NAME  PROPERTY              VALUE                  SOURCE
tank  type                  filesystem             -
tank  creation              Fri Jul 26 21:52 2019  -
tank  used                  1.40T                  -
tank  available             545G                   -
tank  referenced            96K                    -
tank  compressratio         1.23x                  -
tank  mounted               no                     -
tank  quota                 none                   default
tank  reservation           none                   default
tank  recordsize            128K                   default
tank  mountpoint            legacy                 local
tank  sharenfs              off                    default
tank  checksum              on                     default
tank  compression           lz4                    local
tank  atime                 off                    local
tank  devices               on                     default
tank  exec                  on                     default
tank  setuid                on                     default
tank  readonly              off                    default
tank  zoned                 off                    default
tank  snapdir               hidden                 default
tank  aclinherit            restricted             default
tank  createtxg             1                      -
tank  canmount              on                     default
tank  xattr                 on                     default
tank  copies                1                      default
tank  version               5                      -
tank  utf8only              off                    -
tank  normalization         none                   -
tank  casesensitivity       sensitive              -
tank  vscan                 off                    default
tank  nbmand                off                    default
tank  sharesmb              off                    default
tank  refquota              none                   default
tank  refreservation        none                   default
tank  guid                  12411557574271728392   -
tank  primarycache          all                    default
tank  secondarycache        all                    default
tank  usedbysnapshots       0B                     -
tank  usedbydataset         96K                    -
tank  usedbychildren        1.40T                  -
tank  usedbyrefreservation  0B                     -
tank  logbias               latency                default
tank  objsetid              54                     -
tank  dedup                 on                     local
tank  mlslabel              none                   default
tank  sync                  standard               default
tank  dnodesize             legacy                 default
tank  refcompressratio      1.00x                  -
tank  written               96K                    -
tank  logicalused           1.72T                  -
tank  logicalreferenced     42K                    -
tank  volmode               default                default
tank  filesystem_limit      none                   default
tank  snapshot_limit        none                   default
tank  filesystem_count      none                   default
tank  snapshot_count        none                   default
tank  snapdev               hidden                 default
tank  acltype               posixacl               local
tank  context               none                   default
tank  fscontext             none                   default
tank  defcontext            none                   default
tank  rootcontext           none                   default
tank  relatime              off                    default
tank  redundant_metadata    all                    default
tank  overlay               off                    default
tank  encryption            off                    default
tank  keylocation           none                   default
tank  keyformat             none                   default
tank  pbkdf2iters           0                      default
tank  special_small_blocks  0                      default

$ zfs get recordsize tank
NAME  PROPERTY    VALUE    SOURCE
tank  recordsize  128K     default

[cherrymui]

Thanks, @wheelcomplex . The test assumes fallocate is supported on Linux. Apparently this is not true. Maybe we could skip the test in this case.

[gopherbot]

Change https://golang.org/cl/240618 mentions this issue: cmd/link: skip fallocate test if the FS does not support it