(null) in df output

[mrkallenberg]

Description of problem:
When mounting gluster filesystem using a file describing the gluster volume, "null" is shown in the df output.

The exact command to reproduce the issue:

mount /srv/video
df

The full output of the command that failed:

root@dlna02:~# LC_ALL=C df /srv/video
Filesystem        1K-blocks       Used Available Use% Mounted on
glusterfs:(null) 3221225472 3162707484  58517988  99% /srv/video

Expected results:

root@dlna01:~# LC_ALL=C df /srv/video
Filesystem                1K-blocks       Used Available Use% Mounted on
/etc/glusterfs/video.vol 3221225472 3162707484  58517988  99% /srv/video

Mandatory info:
- The output of the gluster volume info command:

root@gluster04:~# gluster volume info video
 
Volume Name: video
Type: Replicate
Volume ID: 8ecda882-a515-4d2d-bcf5-34347b9f58ee
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (2 + 1) = 3
Transport-type: tcp
Bricks:
Brick1: gluster04:/srv/video/brick
Brick2: gluster05:/srv/video/brick
Brick3: gluster06:/srv/video/brick (arbiter)
Options Reconfigured:
cluster.consistent-metadata: on
cluster.granular-entry-heal: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

- The output of the gluster volume status command:

root@gluster04:~# gluster volume status video
Status of volume: video
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick gluster04:/srv/video/brick            52806     0          Y       835  
Brick gluster05:/srv/video/brick            54943     0          Y       842  
Brick gluster06:/srv/video/brick            59513     0          Y       817  
Self-heal Daemon on localhost               N/A       N/A        Y       852  
Self-heal Daemon on gluster05               N/A       N/A        Y       859  
Self-heal Daemon on gluster06               N/A       N/A        Y       834  
 
Task Status of Volume video
------------------------------------------------------------------------------
There are no active volume tasks

- The output of the gluster volume heal command:

root@gluster04:~# gluster volume heal video
Launching heal operation to perform index self heal on volume video has been successful 
Use heal info commands to check status.

**- Provide logs present on following locations of client and server nodes -
/var/log/glusterfs/
Lets take a look at this when something specific is being asked for.

**- Is there any crash ? Provide the backtrace and coredump
No crash

Additional info:
I have collected some information from a working Debian11 system and the same information from the failing Debian12 system. For some of the information I have also tried to install a new version of the glusterfs-client.

- The operating system / glusterfs version:

Server Debian12 (no errors expected on this side)

root@gluster04:~# dpkg --list|grep glusterfs
ii  glusterfs-cli                  10.3-5                                  amd64        clustered file-system (cli package)
ii  glusterfs-client               10.3-5                                  amd64        clustered file-system (client package)
ii  glusterfs-common               10.3-5                                  amd64        GlusterFS common libraries and translator modules
ii  glusterfs-server               10.3-5                                  amd64        clustered file-system (server package)
ii  libglusterfs0:amd64            10.3-5                                  amd64        GlusterFS shared library

Client Debian11 with debian repo: (not seeing the problem with this system)

root@dlna01:~# dpkg --list|grep glusterfs
ii  glusterfs-client                  9.2-1                              amd64        clustered file-system (client package)
ii  glusterfs-common                  9.2-1                              amd64        GlusterFS common libraries and translator modules
ii  libglusterfs0:amd64               9.2-1                              amd64        GlusterFS shared library

Client Debian12 with debian repo:

root@dlna02:~# dpkg --list|grep glusterfs
ii  glusterfs-client                  10.3-5                                  amd64        clustered file-system (client package)
ii  glusterfs-common                  10.3-5                                  amd64        GlusterFS common libraries and translator modules
ii  libglusterfs0:amd64               10.3-5                                  amd64        GlusterFS shared library

Client Debian12 with this repo:

root@dlna02:~# cat /etc/apt/sources.list.d/gluster.list 
deb [signed-by=/usr/share/keyrings/glusterfs-archive-keyring.gpg] https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/12/amd64/apt bookworm main
root@dlna02:~# dpkg --list|grep glusterfs
ii  glusterfs-client                  11.1-1                                  amd64        clustered file-system (client package)
ii  glusterfs-common                  11.1-1                                  amd64        GlusterFS common libraries and translator modules
ii  libglusterfs0:amd64               11.1-1                                  amd64        GlusterFS shared library

The differences I find between Debian11 and Debian12. I get these differences with both the glusterfs-client from the debian repo and from the glusterfs repo. dlna01 is Debian11, dlna02 is Debian12.

dlna01:

root@dlna01:~# grep video /etc/fstab
/etc/glusterfs/video.vol /srv/video glusterfs defaults,_netdev,rw,atime 0 0
root@dlna01:~# LC_ALL=C df /srv/video
Filesystem                1K-blocks       Used Available Use% Mounted on
/etc/glusterfs/video.vol 3221225472 3162707484  58517988  99% /srv/video
root@dlna01:~# grep video /etc/mtab
/etc/glusterfs/video.vol /srv/video fuse.glusterfs rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072 0 0
root@dlna01:~# cat /etc/glusterfs/video.vol 
volume remote1
        type protocol/client
        option transport-type tcp
        option remote-host gluster04
        option remote-subvolume /srv/video/brick
end-volume
 
volume remote2
        type protocol/client
        option transport-type tcp
        option remote-host gluster05
        option remote-subvolume /srv/video/brick
end-volume
 
volume remote3
        type protocol/client
        option transport-type tcp
        option remote-host gluster06
        option remote-subvolume /srv/video/brick
end-volume

volume replicate
        type cluster/replicate
        option arbiter-count 1
        subvolumes remote1 remote2 remote3
end-volume
 
volume writebehind
        type performance/write-behind
        option window-size 1MB
        subvolumes replicate
end-volume
 
volume cache
  type performance/io-cache
  option cache-size 64MB
  subvolumes writebehind
end-volume

dlna02

root@dlna02:~# grep video /etc/fstab
/etc/glusterfs/video.vol /srv/video glusterfs defaults,_netdev,rw,atime 0 0
root@dlna02:~# LC_ALL=C df /srv/video
Filesystem        1K-blocks       Used Available Use% Mounted on
glusterfs:(null) 3221225472 3162707484  58517988  99% /srv/video
root@dlna02:~# grep video /etc/mtab
glusterfs:(null) /srv/video fuse.glusterfs rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072 0 0
root@dlna02:~# cat /etc/glusterfs/video.vol 
volume remote1
        type protocol/client
        option transport-type tcp
        option remote-host gluster04
        option remote-subvolume /srv/video/brick
end-volume
 
volume remote2
        type protocol/client
        option transport-type tcp
        option remote-host gluster05
        option remote-subvolume /srv/video/brick
end-volume
 
volume remote3
        type protocol/client
        option transport-type tcp
        option remote-host gluster06
        option remote-subvolume /srv/video/brick
end-volume

volume replicate
        type cluster/replicate
        option arbiter-count 1
        subvolumes remote1 remote2 remote3
end-volume
 
volume writebehind
        type performance/write-behind
        option window-size 1MB
        subvolumes replicate
end-volume
 
volume cache
  type performance/io-cache
  option cache-size 64MB
  subvolumes writebehind
end-volume

Additional thoughts
It seems that mounting directly towards to a glusterfs node gives the expected result. So it looks like this is only related to mounting by using a file.

root@dlna02:~# LC_ALL=C df /srv/video
Filesystem        1K-blocks       Used Available Use% Mounted on
gluster04:/video 3221225472 3162707484  58517988  99% /srv/video
root@dlna02:~# grep video /etc/mtab
gluster04:/video /srv/video fuse.glusterfs rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072 0 0

[mrkallenberg]

I have tried to put the client in debug mode, and then mount the volume. This is the line from /etc/fstab:

/etc/glusterfs/video.vol /srv/video glusterfs log-level=DEBUG,defaults,_netdev,rw,atime 0 0

Then all logs were deleted and the following was run

root@dlna02:~# mount /srv/video 
root@dlna02:~# LC_ALL=C df /srv/video
Filesystem        1K-blocks       Used Available Use% Mounted on
glusterfs:(null) 3221225472 3162707484  58517988  99% /srv/video
root@dlna02:~# umount /srv/video

Finally a logfile from /var/log/glusterfs/srv-video.log has been attached here.
srv-video.log