Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

internalWebError: operation not supported #6203

Closed
NatoBoram opened this issue Apr 10, 2019 · 7 comments · Fixed by #6680
Closed

internalWebError: operation not supported #6203

NatoBoram opened this issue Apr 10, 2019 · 7 comments · Fixed by #6680
Labels
kind/bug A bug in existing code (including security flaws)

Comments

@NatoBoram
Copy link
Contributor

NatoBoram commented Apr 10, 2019 •
edited
Loading

Version information:

go-ipfs version: 0.4.20-rc1-4fd81e373-dirty
Repo version: 7
System version: amd64/linux
Golang version: go1.12.2

Type: Bug

Description:

When I try to access this hash specifically, I get internalWebError: operation not supported. It also happens with plenty other public nodes.

The hash was found here : ipfs-inactive/package-managers#18 (comment)

Affected

Ok
@NatoBoram NatoBoram mentioned this issue Apr 10, 2019
Closed
@Stebalien
Copy link
Member

Is that hash a symlink?

@magik6k
Copy link
Member

magik6k commented Apr 10, 2019

It's a HAMT sharded directory. Try running ipfs config --json Experimental.ShardingEnabled true and restarting the daemon.

@Stebalien
Copy link
Member

@magik6k doesn't reading a sharded directory work without enabling the sharding experiment? If not, we should probably make it work.

@NatoBoram
Copy link
Contributor Author

Here's parts of my settings.

	"Experimental": {
		"FilestoreEnabled": true,
		"Libp2pStreamMounting": true,
		"P2pHttpProxy": true,
		"QUIC": true,
		"ShardingEnabled": true,
		"UrlstoreEnabled": true
	}
		"DisableBandwidthMetrics": false,
		"DisableNatPortMap": false,
		"DisableRelay": false,
		"EnableAutoNATService": true,
		"EnableAutoRelay": true,
		"EnableRelayHop": true

Pretty much everything else was left as default + badgerds.

@andrew andrew mentioned this issue Apr 16, 2019
Closed
@Stebalien Stebalien added the kind/bug A bug in existing code (including security flaws) label May 15, 2019
@Stebalien
Copy link
Member

Is there a peer providing this file that I could connect to?

@senden9
Copy link

senden9 commented Sep 20, 2019 •
edited
Loading

Hi!
I think I have the same issue.

Bug Description

What I try to do is to implement some kind of mirror of the archlinux package archive.
To do this I created a local mirror of the archlinux archive. This gives me some directories full of symlinks. See /core/cos/x86_64 for example:

$ exa -lha /srv/pacman-mirror/repo/core/os/x86_64 | head
Permissions Size User    Date Modified Name
lrwxrwxrwx    53 stefano 21 Jun  2018  acl-2.2.53-1-x86_64.pkg.tar.xz -> ../../../pool/packages/acl-2.2.53-1-x86_64.pkg.tar.xz
lrwxrwxrwx    57 stefano 21 Jun  2018  acl-2.2.53-1-x86_64.pkg.tar.xz.sig -> ../../../pool/packages/acl-2.2.53-1-x86_64.pkg.tar.xz.sig
lrwxrwxrwx    66 stefano 17 Aug 14:35  amd-ucode-20190815.07b925b-1-any.pkg.tar.xz -> ../../../pool/packages/amd-ucode-20190815.07b925b-1-any.pkg.tar.xz
lrwxrwxrwx    70 stefano 17 Aug 14:35  amd-ucode-20190815.07b925b-1-any.pkg.tar.xz.sig -> ../../../pool/packages/amd-ucode-20190815.07b925b-1-any.pkg.tar.xz.sig
lrwxrwxrwx    66 stefano 29 Aug 20:49  archlinux-keyring-20190827-1-any.pkg.tar.xz -> ../../../pool/packages/archlinux-keyring-20190827-1-any.pkg.tar.xz
lrwxrwxrwx    70 stefano 29 Aug 20:49  archlinux-keyring-20190827-1-any.pkg.tar.xz.sig -> ../../../pool/packages/archlinux-keyring-20190827-1-any.pkg.tar.xz.sig
lrwxrwxrwx    58 stefano  4 Jul 17:11  argon2-20190702-1-x86_64.pkg.tar.xz -> ../../../pool/packages/argon2-20190702-1-x86_64.pkg.tar.xz
lrwxrwxrwx    62 stefano  4 Jul 17:11  argon2-20190702-1-x86_64.pkg.tar.xz.sig -> ../../../pool/packages/argon2-20190702-1-x86_64.pkg.tar.xz.sig
lrwxrwxrwx    54 stefano 21 Jun  2018  attr-2.4.48-1-x86_64.pkg.tar.xz -> ../../../pool/packages/attr-2.4.48-1-x86_64.pkg.tar.xz

I can access this path via the ipfs-webui and see all files listed that are also on my local drive.
image

But I can not look at this folder with one of the following link:

Here I get internalWebError: operation not supported for each of the three requests.

But I can list that files via the cli tool:

$ ipfs ls -v /ipfs/QmZa8Vn963WnVwFKSmZ2Rx9GqHHs4jQpyh1UdAP6W64Us3/repo/core/os/x86_64 | head
Hash                                           Size   Name
QmZ5EpzSj4fngmcnR5DSWkr72AfwSAGKy3ee4MAMxshpoP 0      acl-2.2.53-1-x86_64.pkg.tar.xz
QmVDYD4pcEsbnAhBebC8pCVz5LbAUbKUMgqi9r5pUDcfUF 0      acl-2.2.53-1-x86_64.pkg.tar.xz.sig
QmQNBknk2PeABfHgizJjYwsctcoJPuWAuy3WiJUDEagm56 0      amd-ucode-20190815.07b925b-1-any.pkg.tar.xz
QmfNNUW7WKbqDkA54B3xZv1KW9E4rcqiuvSiChW6HfKjdd 0      amd-ucode-20190815.07b925b-1-any.pkg.tar.xz.sig
QmbexeDpPzCudv26TWCbpjwe6heiCbzcuhtHgULAKp3BQm 0      archlinux-keyring-20190827-1-any.pkg.tar.xz
QmamUDcybMFd6BTRFGmdGvykwNrHoj2cS6mLnduZT4jc7K 0      archlinux-keyring-20190827-1-any.pkg.tar.xz.sig
QmZ7DpDXSXbbfarcJtVuiTS6eff8abjn65JaLdmfZb3yA1 0      argon2-20190702-1-x86_64.pkg.tar.xz
QmeHkEG7dTmN3tLCxD5PbN2swdi2FQo9sHKaH1qoRH9Rm1 0      argon2-20190702-1-x86_64.pkg.tar.xz.sig
QmVmWy7uDf4tSB8rrMiUhGn4SMUC3KJPx8hNRyotpyqeNf 0      attr-2.4.48-1-x86_64.pkg.tar.xz

But I can not look into any of that files.

$ ipfs cat /ipfs/QmZa8Vn963WnVwFKSmZ2Rx9GqHHs4jQpyh1UdAP6W64Us3/repo/core/os/x86_64/acl-2.2.53-1-x86_64.pkg.tar.xz.sig   
Error: operation not supported

$ ipfs object get /ipfs/QmZa8Vn963WnVwFKSmZ2Rx9GqHHs4jQpyh1UdAP6W64Us3/repo/core/os/x86_64/acl-2.2.53-1-x86_64.pkg.tar.xz.sig | json_pp
{
   "Data" : "\b\u0004\u00129../../../pool/packages/acl-2.2.53-1-x86_64.pkg.tar.xz.sig",
   "Links" : []
}

I can look at the direct location where the symlink is pointing to.

$ ipfs cat /ipfs/QmZa8Vn963WnVwFKSmZ2Rx9GqHHs4jQpyh1UdAP6W64Us3/repo/pool/packages/acl-2.2.53-1-x86_64.pkg.tar.xz.sig                                                             :(
(Some binary garbage)

If you want to try a reproduction of that bug(?) feel free to connect to my node via ipfs swarm connect /dns6/stefano-probst.com/tcp/4001/ipfs/QmbNqjDtCbSWgzGzKVDpuj5JaFGSQCRQHQvG5FjweatdoQ :)

Software Versions

$ ipfs diag sys | json_pp
{
   "diskinfo" : {
      "free_space" : 283812909056,
      "fstype" : "2435016766",
      "total_space" : 280463904768
   },
   "environment" : {
      "GOPATH" : "",
      "IPFS_PATH" : ""
   },
   "ipfs_commit" : "4e981576b-dirty",
   "ipfs_version" : "0.4.22",
   "memory" : {
      "swap" : 0,
      "virt" : 2540652000
   },
   "net" : {
      "interface_addresses" : [
         "/ip4/127.0.0.1",
         "/ip4/178.238.228.171",
         "/ip6/::1",
         "/ip6/2a02:c205:2006:6678::1",
         "/ip6/fe80::250:56ff:fe3c:9d17"
      ],
      "online" : true
   },
   "runtime" : {
      "arch" : "amd64",
      "compiler" : "gc",
      "gomaxprocs" : 2,
      "numcpu" : 2,
      "numgoroutines" : 5983,
      "os" : "linux",
      "version" : "go1.12.8"
   }
}

$ uname -a
Linux srv01 5.2.11-arch1-1-ARCH #1 SMP PREEMPT Thu Aug 29 08:09:36 UTC 2019 x86_64 GNU/Linux

@Stebalien
Copy link
Member

Ah. So, the issue is that:

  1. That directory contains a bunch of symlinks.
  2. Those symlinks don't have sizes.
  3. We error out because a file doesn't have a size.

@Stebalien Stebalien mentioned this issue Sep 26, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug A bug in existing code (including security flaws)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

improve gateway symlink handling ipfs/kubo
4 participants
@Stebalien @senden9 @magik6k @NatoBoram