Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

C++ package manager request tracking issue #4579

Open
Wumpf opened this issue Dec 18, 2023 · 8 comments
Open

C++ package manager request tracking issue #4579

Wumpf opened this issue Dec 18, 2023 · 8 comments
Labels
🌊 C++ API C/C++ API specific 🎄 tracking issue issue that tracks a bunch of subissues

Comments

@Wumpf
Copy link
Member

Wumpf commented Dec 18, 2023 •
edited
Loading

For the moment we decided to not support additional build systems and package management for C++, since there's simply too many to choose from and this becomes a bit of a maintenance burden that we're not ready for yet.

This ticket collects the various requests that came up so far and what state they're in:
@Wumpf Wumpf added other Generated by the "Other" issue template 👀 needs triage This issue needs to be triaged by the Rerun team labels Dec 18, 2023
This was referenced Dec 18, 2023
Closed
Closed
Closed
@Wumpf Wumpf added 🌊 C++ API C/C++ API specific and removed other Generated by the "Other" issue template 👀 needs triage This issue needs to be triaged by the Rerun team labels Dec 18, 2023
@abey79 abey79 added the 🎄 tracking issue issue that tracks a bunch of subissues label Dec 19, 2023
@rgolovanov
Copy link
Contributor

rgolovanov commented Jan 22, 2024 •
edited
Loading

@abey79 , @Wumpf I've started work on conan recipe (#4420 ) in our local conan system and faced a few things. Maybe, we could discuss it here or in discord or have a call. Here are the topics:

C++ SDK provided in assets has good and clear process of integration with just couple lines of code in CMake, but it seems to be very important that own code is compiled along with some Rerun sources (*.c - files). However, the classical package should contain only pre-built libraries (lib or dll) and header files. I tried to create conan package and it contains only hpp files with these libs:

  • arrow_bundled_dependencies.lib
  • arrow_static.lib
  • rerun_c__win_x64.lib (from official SDK asset)
  • rerun_sdk.lib + rerun_sdk.pdb

Next, when I try to link rerun conan package to existing project XXXX I'm getting these linker errors - apparently libs don't contain all code:

[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.05.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.00.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.12.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.01.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAStartup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSACleanup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_freeaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_shutdown [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAGetLastError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_setsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSARecv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASend [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_send [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_accept [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_ioctlsocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getpeername [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSADuplicateSocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtCreateFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_RtlNtStatusToDosError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtReadFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtWriteFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_connect [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_select [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recvfrom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(getrandom-b857062dfe0c4b33.getrandom.2ae02711c9719d53-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_bind [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_listen [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockname [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_sendto [XXXXXXXXXX.vcxproj]
[build] XXXXX.exe : fatal error LNK1120: 30 unresolved externals [XXXXXXXXXX.vcxproj]

I would like to understand better the structure of C++ code, how to make it compilable into self-sufficient library and how did you get rerun_c__win_x64.lib - what does it contain in practice.

I'm thinking of maybe forcing linker to produce all symbols by adding following options (not sure it will resolve all linker problems though):

MSVC: /WHOLEARCHIVE option in the linker.
GCC use the -Wl,--whole-archive linker option.

What do you think?

@Wumpf
Copy link
Member Author

Wumpf commented Jan 23, 2024

@rgolovanov the missing symbols you hit there are from a bunch of very common system dependencies. These to be accurate ws2_32.dll Bcrypt.dll Userenv.dll ntdll.dll
Afaik whole-archive linking is generally discouraged since it can lead to a lot of bloat (everyone else now using anything from the windows system comes with their own winsock & ntdll etc.). I'm not familiar with Conan, but I'd assume that these system libraries would be explicitly specified as a dependency whenever linking with the static version of Rerun.

@rgolovanov
Copy link
Contributor

@Wumpf thanks for comment. Yes, now I see it should come from here but apparently was cutoff during conan package creation. I'll check what can be done 👍

This was referenced Feb 28, 2024
Closed
Merged
Wumpf pushed a commit that referenced this issue Mar 4, 2024
@traversaro
Add RERUN_INSTALL_RERUN_C CMake option to disable installation of rer…
1b4aa3d 
…un_c library (#5374)

When building the C++ SDK with the `BUILD_SHARED_LIBS` option set to
`ON`, it is not compulsory to also install the `rerun_c` static library.
This PR adds a CMake option `RERUN_INSTALL_RERUN_C` (enabled by
default), that can be disabled when `rerun_sdk` is not compiled as
static library.

This is useful when packaging the rerun C++ SDK if you want to avoid to
also install the rerun_c static library (related to
#4579).

### What

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5374/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5374/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5374/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5374)
- [Docs
preview](https://rerun.io/preview/538c4c8eaad0963eaa2a6915c1179106d5acc111/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/538c4c8eaad0963eaa2a6915c1179106d5acc111/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
@Wumpf Wumpf mentioned this issue Mar 15, 2024
Closed
@traversaro
Copy link
Contributor

For what regards C++ SDK support for conda-forge, this is now solved (at least on Linux/macOS) with the librerun-sdk package, see conda-forge/staged-recipes#25648 and https://github.com/conda-forge/librerun-sdk-feedstock .

@Wumpf
Copy link
Member Author

Wumpf commented Mar 18, 2024

@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?

@traversaro
Copy link
Contributor

@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?

Sure, there are a bit of aspects that I would to tackle before (support for it in the opencv/eigen example, Windows support, general dogfooding it to understand if it works fine), and then I think we can can add it there, does it make sense?

@Wumpf
Copy link
Member Author

Wumpf commented Mar 18, 2024

Of course, whenever you feel comfortable with it :)

@traversaro
Copy link
Contributor

Of course, whenever you feel comfortable with it :)

Now that we have the Windows package as well, I think it is fine to add it: #6381 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🌊 C++ API C/C++ API specific 🎄 tracking issue issue that tracks a bunch of subissues
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Wumpf @traversaro @rgolovanov @abey79