Package layout

[isuruf]

Comment:

Just a suggestion to make cross-compilation work

- cuda-nvcc-tools
   - arch dependent package
   - files:
     - targets/<arch>/bin/nvcc   (no other executables in this directory)
     - bin/nvcc -> targets/<arch>/bin/nvcc (symlink)
     - bin/*
     - nvvm/bin/cicc   (this can go into `bin/` as well)
   
- cuda-nvcc-dev_{{ target_platform }}
   - noarch: generic package
   - files:
     - targets/<arch>/include
     - targets/<arch>/lib
     - targets/<arch>/nvvm/include
     - targets/<arch>/nvvm/libdevice
     - targets/<arch>/nvvm/lib

- cuda-nvcc
  - files:
    - lib/* -> targets/<arch>/lib/*
    - include/* -> targets/<arch>/include/*
  - requirements:
     - cuda-nvcc-dev_{{ target_platform }}
     - cuda-nvcc-tools

These need to go on a different feedstock

- cuda-nvcc_{{ cross_target_platform }}
  - Meta-package that has the activation script
  - requirements:
     - cuda-nvcc-tools
     - cuda-nvcc-dev_{{ cross_target_platform }}
     - cuda-nvcc      (if native. needed for CMake)
  - files:
      - etc/conda/activate.d/
      - etc/conda/deactivate.d/

cc @robertmaynard, @jakirkham, @conda-forge/core

[isuruf]

cc @adibbley

[robertmaynard]

Where would executables like __nvcc_device_query, fatbinary, and cudafe++ that nvcc calls internally reside with this proposal?

[isuruf]

They can be in bin/ as they don't seem like they would interfere. If you think that's a problem, they should go in libexec/

[bdice]

Where would activation scripts live? Currently we are using an activation script to provide environment variables that help nvcc find the host compiler. https://github.com/conda-forge/cuda-nvcc-feedstock/blob/f11fd42b65995da716990ef5c68487c2a2f6cd6a/recipe/activate.sh

What does the top level cuda-nvcc package include / depend on?

[robertmaynard]

Can you post what env variable changes are needed with the this proposal to get CMake to work as well @isuruf?

[robertmaynard]

The split of nvvm into nvvm/bin and targets/<arch>/nvvm/[include/libdevice/lib] isn't needed to me. All the nvvm components are host only and don't change based on what target you are cross compiling for. So they should all stay under ${PREFIX}/nvvm/

[isuruf]

Where would activation scripts live? Currently we are using an activation script to provide environment variables that help nvcc find the host compiler. https://github.com/conda-forge/cuda-nvcc-feedstock/blob/f11fd42b65995da716990ef5c68487c2a2f6cd6a/recipe/activate.sh

Same as now. etc/conda/{de,}activate.d

What does the top level cuda-nvcc package include / depend on?

Depends on if you want the activation script to be a part or not. This will only be a user facing package and not used by conda-build at all.

Can you post what env variable changes are needed with the this proposal to get CMake to work as well @isuruf?

No env variable changes except NVCC_PREPEND_FLAGS.

All the nvvm components are host only and don't change based on what target you are cross compiling for. So they should all stay under ${PREFIX}/nvvm/

Really? There are pacakges that include nvvm.h and link to libnvvm.so right?

[jakirkham]

Yeah the current NVVM path is used by Numba

[robertmaynard]

All the nvvm components are host only and don't change based on what target you are cross compiling for. So they should all stay under ${PREFIX}/nvvm/

Really? There are pacakges that include nvvm.h and link to libnvvm.so right?

I was primarily basing this on what the cuda toolkit cross compilation packages like https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda-repo-cross-sbsa-ubuntu2004-12-1-local_12.1.1-1_all.deb provide. Specifically the cuda-nvcc package inside of that only provides:

├── usr
│   ├── local
│   │   └── cuda-12.2
│   │       └── targets
│   │           └── sbsa-linux
│   │               ├── include
│   │               │   ├── crt
│   │               │   │   ├── common_functions.h
│   │               │   │   ├── cudacc_ext.h
│   │               │   │   ├── device_double_functions.h
│   │               │   │   ├── device_double_functions.hpp
│   │               │   │   ├── device_functions.h
│   │               │   │   ├── device_functions.hpp
│   │               │   │   ├── func_macro.h
│   │               │   │   ├── host_config.h
│   │               │   │   ├── host_defines.h
│   │               │   │   ├── host_runtime.h
│   │               │   │   ├── math_functions.h
│   │               │   │   ├── math_functions.hpp
│   │               │   │   ├── mma.h
│   │               │   │   ├── mma.hpp
│   │               │   │   ├── nvfunctional
│   │               │   │   ├── sm_70_rt.h
│   │               │   │   ├── sm_70_rt.hpp
│   │               │   │   ├── sm_80_rt.h
│   │               │   │   ├── sm_80_rt.hpp
│   │               │   │   ├── sm_90_rt.h
│   │               │   │   ├── sm_90_rt.hpp
│   │               │   │   └── storage_class.h
│   │               │   ├── fatbinary_section.h
│   │               │   └── nvPTXCompiler.h
│   │               └── lib
│   │                   └── libnvptxcompiler_static.a

So nvvm should be considered to be a host only tool.

[adibbley]

I've started to draft PR #13 following the proposal here. A few followup questions:

• Why the naming of cuda-nvcc-tools instead of sticking with cuda-nvcc?
• What is the use case for cuda-nvcc-impl? Should we just move these files to cuda-nvcc?

[isuruf]

cuda-nvcc-tools as proposed above is incomplete without the other packages. If cuda-nvcc is supposed to be used by the user, then it should be complete.
Symlinks in cuda-nvcc-impl are needed for cmake to work properly.

Should we just move these files to cuda-nvcc?

What do you want cuda-nvcc package to be?

[adibbley]

Ideally cuda-nvcc would be the complete package for the host and the relevant _{{ cross_target_platform }} package would enable cross. This would be similar to how we handle it for debians, not sure if that is oversimplifying things here though.

[isuruf]

Do you also want the activation scripts in it or not?

[adibbley]

I would think so, although I'm not tied to that if there is a better place for them.

[isuruf]

I mean, do you want cuda-nvcc to include them directly or indirectly? Or do you want cuda-nvcc not have env variables at all like what we do for gcc, gxx meta-packages?

[adibbley]

There is a suggestion to put them in cuda-nvcc_{{ target_platform }}: #13 (comment). That is also fine by me.

[adibbley]

Just seeing the edits to the OP here. Following this, can activation scripts be directly in cuda-nvcc?

Alternatively, would it work to rename the above cuda-nvcc back to cuda-nvcc-impl and use cuda-nvcc just for activation as:

- cuda-nvcc
  - requirements:
     - cuda-nvcc-impl
  - files:
      - etc/conda/activate.d/
      - etc/conda/deactivate.d/

[isuruf]

Following this, can activation scripts be directly in cuda-nvcc?

We could. I was thinking some people might prefer to use cuda-nvcc without the activation scripts.

Alternatively, would it work to rename the above cuda-nvcc back to cuda-nvcc-impl and use cuda-nvcc just for activation as:

Sure, that would work as well. Then people who prefer to use cuda-nvcc without activation scripts can just use cuda-nvcc-impl.

[adibbley]

Great. In that case, does it make sense to split the feedstock as:

cuda-nvcc-feedstock:

cuda-nvcc_{{ cross_target_platform }}
cuda-nvcc

cuda-nvcc-impl-feedstock:

cuda-nvcc-tools
cuda-nvcc-dev_{{ target_platform }}
cuda-nvcc-impl

[isuruf]

Makes sense.

[adibbley]

Opened conda-forge/staged-recipes#22802 to match this