cannot download with wget

[goi42]

If I try to access
https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
from a browser, I can download it without issues,
but if I try with wget:

wget --spider https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz

I get 404 Not Found:

Spider mode enabled. Check if remote file exists.
--2022-06-30 11:54:56--  https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
Resolving pypi.io (pypi.io)... 151.101.128.223, 151.101.192.223, 151.101.64.223, ...
Connecting to pypi.io (pypi.io)|151.101.128.223|:443... connected.
HTTP request sent, awaiting response... 301 Redirect to Primary Domain
Location: https://pypi.org/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz [following]
Spider mode enabled. Check if remote file exists.
--2022-06-30 11:54:56--  https://pypi.org/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
Resolving pypi.org (pypi.org)... 151.101.64.223, 151.101.0.223, 151.101.128.223, ...
Connecting to pypi.org (pypi.org)|151.101.64.223|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://files.pythonhosted.org/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz [following]
Spider mode enabled. Check if remote file exists.
--2022-06-30 11:54:56--  https://files.pythonhosted.org/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 146.75.77.63, 2a04:4e42:83::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|146.75.77.63|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
Remote file does not exist -- broken link!!!

[jpivarski]

I don't know what you're trying to do here. The normal way to install a package like this is

# --pre required to get latest pre-release
pip install --pre fastjet

but even if you're not doing that and want to download it directly, the URL hosted by PyPI for the source tarball is https://files.pythonhosted.org/packages/aa/93/4f4d6ae1b4b7da5e432abb1ac7e21fc53b2752ad23b21aa1ef660bb69721/fastjet-3.3.4.0rc9.tar.gz.

I don't know where https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz comes from. I can't download anything from that URL, even with my browser. This package is named with all lower-case characters, so I don't know how your URL got the "FastJet" (capital F, capital J) in it.

If you need an HTTP snapshot of the git repo, which is a little different from the source distribution that you would normally access via pip install --pre fastjet, GitHub does supply one: https://github.com/scikit-hep/fastjet/archive/refs/tags/v3.3.4.0rc9.tar.gz. But if you wanted the git repo, the normal way to get that would be

# --recursive required to get C++ dependencies
git clone --recursive https://github.com/scikit-hep/fastjet.git

Although wget could be involved in installation, via some non-standard method, I don't see how you chose that URL.

[goi42]

The URL came up in the context of

conda skeleton pypi FastJet --version 3.3.4.0rc9 --extra-specs pybind11
conda-build fastjet --use-local --override-channels

which was recommended as a way to convert pip packages for use in conda environments. (ASFAIK, FastJet is not on conda-forge.)

It is confusing to me that my browser can access it while yours does not... very strange.

[goi42]

Full output of conda-build fastjet --use-local --override-channels:

No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for fastjet
INFO:conda_build.metadata:Attempting to finalize metadata for fastjet
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/noarch/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/linux-64/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/noarch/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/channeldata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /bioconda/channeldata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/channeldata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/channeldata.json HTTP/1.1" 200 None
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
BUILD START: ['fastjet-3.3.4.0rc9-py39_0.tar.bz2']
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /data/home/michaelkw/miniconda/conda-bld/fastjet_1656614919286/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place


The following NEW packages will be INSTALLED:

    _libgcc_mutex:    0.1-conda_forge           conda-forge
    _openmp_mutex:    4.5-2_gnu                 conda-forge
    bzip2:            1.0.8-h7f98852_4          conda-forge
    ca-certificates:  2022.6.15-ha878542_0      conda-forge
    ld_impl_linux-64: 2.36.1-hea4e1c9_2         conda-forge
    libffi:           3.4.2-h7f98852_5          conda-forge
    libgcc-ng:        12.1.0-h8d9b700_16        conda-forge
    libgomp:          12.1.0-h8d9b700_16        conda-forge
    libnsl:           2.0.0-h7f98852_0          conda-forge
    libuuid:          2.32.1-h7f98852_1000      conda-forge
    libzlib:          1.2.12-h166bdaf_1         conda-forge
    ncurses:          6.3-h27087fc_1            conda-forge
    openssl:          3.0.4-h166bdaf_2          conda-forge
    pip:              22.1.2-pyhd8ed1ab_0       conda-forge
    python:           3.9.13-h2660328_0_cpython conda-forge
    python_abi:       3.9-2_cp39                conda-forge
    readline:         8.1.2-h0f457ee_0          conda-forge
    setuptools:       62.6.0-py39hf3d152e_0     conda-forge
    sqlite:           3.39.0-h4ff8645_0         conda-forge
    tk:               8.6.12-h27826a3_0         conda-forge
    tzdata:           2022a-h191b570_0          conda-forge
    wheel:            0.37.1-pyhd8ed1ab_0       conda-forge
    xz:               5.2.5-h516909a_1          conda-forge
    zlib:             1.2.12-h166bdaf_1         conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Source cache directory is: /data/home/michaelkw/miniconda/conda-bld/src_cache
INFO:conda_build.source:Source cache directory is: /data/home/michaelkw/miniconda/conda-bld/src_cache
Downloading source to cache: FastJet-3.3.4.0rc9_6121b3e56a.tar.gz
INFO:conda_build.source:Downloading source to cache: FastJet-3.3.4.0rc9_6121b3e56a.tar.gz
Downloading https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
INFO:conda_build.source:Downloading https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.io:443
DEBUG:urllib3.connectionpool:https://pypi.io:443 "GET /packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz HTTP/1.1" 301 122
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz HTTP/1.1" 301 279
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz HTTP/1.1" 404 0
Error: HTTP 404 NOT FOUND for url <https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz>
Elapsed: 00:00.224494

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
WARNING:conda_build.source:Error: HTTP 404 NOT FOUND for url <https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz>
Elapsed: 00:00.224494

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
Could not download https://pypi.io/packages/source/F/FastJet/FastJet-3.3.4.0rc9.tar.gz

[jpivarski]

@chrisburr and @henryiii may help us get this library on conda-forge, but they'll probably want to wait for it to get into non-pre-release, first. As I said in #92, we'll need a maintainer to address user feedback like yours and some compilation issues that are holding it back. There's also a new version of FastJet (capital F, J) that we should adopt, but that will require some work, too.

All in all, since this is getting regular users now, we need to draw attention to it to find someone who can maintain it.

[jmduarte]

Closing this issue for now, but please feel free to open a new one on adding fastjet to conda-forge :)