Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hctl mapping for SATA drives only #14

Closed
dieneuser opened this issue Jun 20, 2024 · 9 comments · Fixed by #15
Closed

hctl mapping for SATA drives only #14

dieneuser opened this issue Jun 20, 2024 · 9 comments · Fixed by #15

Comments

@dieneuser
Copy link

I have a DXP4800 and am currently working on the mapping.

The script uses lsblk in line 98 for the mapping - unfortunately it does not take into account whether other drives are connected.
So I have also connected a USB drive for Unraid and a DVD ROM, which of course also receive hctl numbers starting at 0:0:0:0.

The result is then

root@nas:~# lsblk -S -o name,hctl,tran | tail -n +2
sda  1:0:0:0    usb
sdb  2:0:0:0    sata
sdc  3:0:0:0    sata
sdd  4:0:0:0    sata
sde  5:0:0:0    sata
sr0  0:0:0:0    usb

My suggestion would be to extend the lsblk command in line 98 and really only extract the existing SATA drives:

root@nas:~# lsblk -S -o name ,hctl, tran | sed -n 's/\(sd.\)  \([0-9]:[0-9]:[0-9]:[0-9]\).*sata$/\1 \2/p'
sdb 2:0:0:0
sdc 3:0:0:0
sdd 4:0:0:0
sde 5:0:0:0
@miskcoo
Copy link
Owner

miskcoo commented Jun 20, 2024

Not sure if this is the same as #9 . I am wondering that

  1. Are 2:0:0:0 - 5:0:0:0 mapped to disk1 to disk4 in order?
  2. Will this map and the HCTL change after rebooting, regardless of whether the USB exists?

If the map is correct and does not change, then we may change the hctl_map array. However, if the HCTL becomes 0:0:0:0 to 3:0:0:0 when you remove the two USB devices, it may be difficult to determine how to map the disk. In such a case, I am wondering what would happen if you remove the disk in the first slot. Would that be the following, i.e., 2:0:0:0 still corresponds to disk1?

sdX  1:0:0:0    usb
sdX  3:0:0:0    sata
sdX  4:0:0:0    sata
sdX  5:0:0:0    sata
sr0  0:0:0:0    usb

@miskcoo
Copy link
Owner

miskcoo commented Jun 20, 2024

By the way, can you provide the outputs of lsblk -ahl /sys/block | grep ata and ls -ahl /sys/bus/scsi/devices/?

@yooooki
Copy link
Contributor

yooooki commented Jun 20, 2024

I think it might be a good idea to instruct users to record serial number before installment, and configure the serial numbers in the script. Many models labels have their serial number printed on the label, it take only several minutes to record them and write into scripts, and solve the mapping problem once for ever.

@dieneuser
Copy link
Author

Are 2:0:0:0 - 5:0:0:0 mapped to disk1 to disk4 in order?

Yes, the order of disk to the (LED) number is correct.

Will this map and the HCTL change after rebooting, regardless of whether the USB exists?

As soon as I remove the USB DVD drive, the numbering is different.

root@nas:~# lsblk -S -o name,hctl,tran
NAME HCTL       TRAN
sda  0:0:0:0    usb
sdb  1:0:0:0    sata
sdc  2:0:0:0    sata
sdd  3:0:0:0    sata
sde  4:0:0:0    sata

I think that all connected devices on the SOC are numbered first and then all external chipsets (ASMedia...).
This means that the order cannot actually be statically configured.

By the way, can you provide the outputs of lsblk -ahl /sys/block | grep ata and ls -ahl /sys/bus/scsi/devices/?

root@nas:~# lsblk -ahl /sys/block | grep ata
    DISC-ZERO  discard zeroes data

root@nas:~# ls -ahl /sys/bus/scsi/devices/
total 0
drwxr-xr-x 2 root root 0 Jun 17 13:02 ./
drwxr-xr-x 4 root root 0 Jun 17 13:02 ../
lrwxrwxrwx 1 root root 0 Jun 17 13:02 0:0:0:0 -> ../../../devices/pci0000:00/0000:00:0d.0/usb2/2-1/2-1:1.0/host0/target0:0:0/0:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 1:0:0:0 -> ../../../devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3:1.0/host1/target1:0:0/1:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 20 10:22 26:0:0:0 -> ../../../devices/pci0000:00/0000:00:14.0/usb4/4-3/4-3.3/4-3.3:1.0/host26/target26:0:0/26:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 2:0:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata1/host2/target2:0:0/2:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 3:0:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata2/host3/target3:0:0/3:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 4:0:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host4/target4:0:0/4:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 5:0:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata4/host5/target5:0:0/5:0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host0 -> ../../../devices/pci0000:00/0000:00:0d.0/usb2/2-1/2-1:1.0/host0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host1 -> ../../../devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3:1.0/host1/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host10 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata9/host10/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host11 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata10/host11/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host12 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata11/host12/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host13 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata12/host13/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host14 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata13/host14/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host15 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata14/host15/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host16 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata15/host16/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host17 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata16/host17/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host18 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata17/host18/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host19 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata18/host19/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host2 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata1/host2/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host20 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata19/host20/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host21 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata20/host21/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host22 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata21/host22/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host23 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata22/host23/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host24 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata23/host24/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host25 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata24/host25/
lrwxrwxrwx 1 root root 0 Jun 20 16:58 host26 -> ../../../devices/pci0000:00/0000:00:14.0/usb4/4-3/4-3.3/4-3.3:1.0/host26/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host3 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata2/host3/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host4 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host4/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host5 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata4/host5/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host6 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata5/host6/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host7 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata6/host7/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host8 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata7/host8/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 host9 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata8/host9/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target0:0:0 -> ../../../devices/pci0000:00/0000:00:0d.0/usb2/2-1/2-1:1.0/host0/target0:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target1:0:0 -> ../../../devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3:1.0/host1/target1:0:0/
lrwxrwxrwx 1 root root 0 Jun 20 16:58 target26:0:0 -> ../../../devices/pci0000:00/0000:00:14.0/usb4/4-3/4-3.3/4-3.3:1.0/host26/target26:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target2:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata1/host2/target2:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target3:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata2/host3/target3:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target4:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host4/target4:0:0/
lrwxrwxrwx 1 root root 0 Jun 17 13:02 target5:0:0 -> ../../../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata4/host5/target5:0:0/

In my model, all sata disks are connected to an ASMedia controller:

01:00.0 SATA controller: ASMedia Technology Inc. ASM1164 Serial ATA AHCI Controller (rev 02)
[2:0:0:0]    disk    ATA      WDC WD80EFAX-68K 0A81  /dev/sdb   8.00TB
[3:0:0:0]    disk    ATA      WDC WD40EFRX-68N 0A82  /dev/sdc   4.00TB
[4:0:0:0]    disk    ATA      Samsung SSD 870  1B6Q  /dev/sdd   1.00TB
[5:0:0:0]    disk    ATA      WDC WD80EFBX-68A 0A85  /dev/sde   8.00TB

I think it might be a good idea to instruct users to record serial number before installment, and configure the serial numbers in the script. Many models labels have their serial number printed on the label, it take only several minutes to record them and write into scripts, and solve the mapping problem once for ever.

That's a good idea - at least it would allow every user to define the appropriate configuration for all models in all variants.

miskcoo added a commit that referenced this issue Jun 20, 2024
@miskcoo
Copy link
Owner

miskcoo commented Jun 20, 2024

I just noticed that UGOS use the ata number to map the disk, and it seems to work better then hctl:

# ATA Mapping
ls -ahl /sys/block | awk '{
    if (match($0, /ata[0-9]+/)) {
        ata = substr($0, RSTART, RLENGTH);
        if (match($0, /[^\/]+$/)) {
            basename = substr($0, RSTART, RLENGTH);
        }
        print basename, ata;
    }
}'

# HCTL Mapping
lsblk -S -o name,hctl -x hctl

@dieneuser Maybe this would work. Do you want to test it? (see this branch for the new script)

I think it might be a good idea to instruct users to record serial number before installment, and configure the serial numbers in the script. Many models labels have their serial number printed on the label, it take only several minutes to record them and write into scripts, and solve the mapping problem once for ever.

Good idea. It is definitely a better solution. I think we can add a config file about that.

@dieneuser
Copy link
Author

dieneuser commented Jun 20, 2024

I just noticed that UGOS use the ata number to map the disk, and it seems to work better then hctl:

# ATA Mapping
ls -ahl /sys/block | awk '{
    if (match($0, /ata[0-9]+/)) {
        ata = substr($0, RSTART, RLENGTH);
        if (match($0, /[^\/]+$/)) {
            basename = substr($0, RSTART, RLENGTH);
        }
        print basename, ata;
    }
}'

# HCTL Mapping
lsblk -S -o name,hctl -x hctl

@dieneuser Maybe this would work. Do you want to test it? (see this branch for the new script)

These are the results from your new script:

root@nas:~# # ATA Mapping
ls -ahl /sys/block | awk '{
    if (match($0, /ata[0-9]+/)) {
        ata = substr($0, RSTART, RLENGTH);
        if (match($0, /[^\/]+$/)) {
            basename = substr($0, RSTART, RLENGTH);
        }
        print basename, ata;
    }
}'
 ata1
 ata2
 ata3
 ata4

root@nas:~# # HCTL Mapping
lsblk -S -o name,hctl -x hctl
NAME HCTL
sda  0:0:0:0
sr0  1:0:0:0
sdb  2:0:0:0
sdc  3:0:0:0
sdd  4:0:0:0
sde  5:0:0:0

But something is wrong with the regex, the basename is not found.

Here is the output from the ls command:

root@nas:~# ls -ahl /sys/block
total 0
drwxr-xr-x  2 root root 0 Jun 20 17:23 ./
dr-xr-xr-x 13 root root 0 Jun 20 17:23 ../
lrwxrwxrwx  1 root root 0 Jun 20 17:25 dm-0 -> ../devices/virtual/block/dm-0/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 dm-1 -> ../devices/virtual/block/dm-1/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 dm-2 -> ../devices/virtual/block/dm-2/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 dm-3 -> ../devices/virtual/block/dm-3/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 dm-4 -> ../devices/virtual/block/dm-4/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop0 -> ../devices/virtual/block/loop0/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop1 -> ../devices/virtual/block/loop1/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop2 -> ../devices/virtual/block/loop2/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop3 -> ../devices/virtual/block/loop3/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop4 -> ../devices/virtual/block/loop4/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop5 -> ../devices/virtual/block/loop5/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop6 -> ../devices/virtual/block/loop6/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 loop7 -> ../devices/virtual/block/loop7/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 md1p1 -> ../devices/virtual/block/md1p1/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 md2p1 -> ../devices/virtual/block/md2p1/
lrwxrwxrwx  1 root root 0 Jun 20 17:25 md3p1 -> ../devices/virtual/block/md3p1/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 nvme0n1 -> ../devices/pci0000:00/0000:00:1d.0/0000:04:00.0/nvme/nvme0/nvme0n1/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 nvme1n1 -> ../devices/pci0000:00/0000:00:1d.2/0000:05:00.0/nvme/nvme1/nvme1n1/
lrwxrwxrwx  1 root root 0 Jun 20 17:23 sda -> ../devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3:1.0/host0/target0:0:0/0:0:0:0/block/sda/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 sdb -> ../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata1/host2/target2:0:0/2:0:0:0/block/sdb/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 sdc -> ../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata2/host3/target3:0:0/3:0:0:0/block/sdc/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 sdd -> ../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host4/target4:0:0/4:0:0:0/block/sdd/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 sde -> ../devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata4/host5/target5:0:0/5:0:0:0/block/sde/
lrwxrwxrwx  1 root root 0 Jun 20 17:24 sr0 -> ../devices/pci0000:00/0000:00:0d.0/usb2/2-1/2-1:1.0/host1/target1:0:0/1:0:0:0/block/sr0/

@miskcoo
Copy link
Owner

miskcoo commented Jun 20, 2024

Sorry, forgot to remove the trailing slash:

        ls -ahl /sys/block | sed 's/\/$//' | awk '{
            if (match($0, /ata[0-9]+/)) {
                ata = substr($0, RSTART, RLENGTH);
                if (match($0, /[^\/]+$/)) {
                    basename = substr($0, RSTART, RLENGTH);
                }
                print basename, ata;
            }
        }'

@dieneuser
Copy link
Author

dieneuser commented Jun 20, 2024

Sorry, forgot to remove the trailing slash:

        ls -ahl /sys/block | sed 's/\/$//' | awk '{
            if (match($0, /ata[0-9]+/)) {
                ata = substr($0, RSTART, RLENGTH);
                if (match($0, /[^\/]+$/)) {
                    basename = substr($0, RSTART, RLENGTH);
                }
                print basename, ata;
            }
        }'
sdb ata1
sdc ata2
sdd ata3
sde ata4

Now the mapping looks very good.
It now works correctly even when I work without a DVD drive or with/without other USB drives.

the mapping as one liner: ls -ahl /sys/block | sed -n 's/.*\/\(ata[0-9]\)\/.*\/\(.*\)\/$/\2 \1/p'

sdb ata1
sdc ata2
sdd ata3
sde ata4

@miskcoo
Copy link
Owner

miskcoo commented Jun 20, 2024

I see. I will merge the new PR and set ATA as the default mapping if it also works in other models.

miskcoo added a commit that referenced this issue Jun 21, 2024
1. Add the config file /etc/ugreen-leds.conf for environment variables used in ugreen-diskiomon (see scripts/ugreen-leds.conf)
2. Make the blinking of netdev activities, colors of different disk states / NIC link speeds configurable
3. Add the check of zpool health
4. Add the check of gateway connectivity and the link speed of network device
5. Split the systemd service to ugreen-diskiomon.service and ugreen-netdevmon@<device name>.service
6. HCTL mapping will fail in some cases (hctl mapping for SATA drives only #14), UGOS uses ATA number to map disks to slots. I changed the default mapping to ATA, which can be shown by the following scripts:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants