Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nfnetfilter log #1571

Merged
merged 3 commits into from
Nov 20, 2019
Merged

Nfnetfilter log #1571

merged 3 commits into from
Nov 20, 2019

Conversation

vorner
Copy link
Contributor

@vorner vorner commented Oct 28, 2019

This adds the constants from linux/netfilter/nfnetlink.h and nfnetlink_log.h. These are the files I need for jbaublitz/neli#48. After this gets in, I'd like to follow-up with the other nfnetlink_*.h files too, as I'd like to extend neli with further protocols in the future, but I want to do a smaller PR first to see if there are some things to tweak.

I've noticed similar netfilter constants are also in the android subfolder, therefore I'm adding them there too (I don't like the copy-pasting, but it seems the other ones are already copy-pasted). I assume the test will catch it if anything is different on that platform.

@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @gnzlbg (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@vorner
Copy link
Contributor Author

vorner commented Oct 28, 2019

I'm getting some CI errors and I'd ask for little advice how to proceed. All the musl based ones fail with something like this:

cargo:warning=/checkout/target/aarch64-unknown-linux-musl/debug/build/libc-test-914b8fd8030c9123/out/main.c:26028:65: error: 'NFNLGRP_NFTRACE' undeclared here (not in a function); did you mean 'NFNLGRP_NFTABLES'?
cargo:warning=             static const int __test_const_NFNLGRP_NFTRACE_val = NFNLGRP_NFTRACE;
cargo:warning=                                                                 ^~~~~~~~~~~~~~~
cargo:warning=                                                                 NFNLGRP_NFTABLES
cargo:warning=/checkout/target/aarch64-unknown-linux-musl/debug/build/libc-test-914b8fd8030c9123/out/main.c:26118:67: error: 'NFNL_BATCH_UNSPEC' undeclared here (not in a function); did you mean 'NFTA_NAT_UNSPEC'?
cargo:warning=             static const int __test_const_NFNL_BATCH_UNSPEC_val = NFNL_BATCH_UNSPEC;
cargo:warning=                                                                   ^~~~~~~~~~~~~~~~~
cargo:warning=                                                                   NFTA_NAT_UNSPEC
cargo:warning=/checkout/target/aarch64-unknown-linux-musl/debug/build/libc-test-914b8fd8030c9123/out/main.c:26124:66: error: 'NFNL_BATCH_GENID' undeclared here (not in a function); did you mean 'NFNL_MSG_BATCH_END'?
cargo:warning=             static const int __test_const_NFNL_BATCH_GENID_val = NFNL_BATCH_GENID;
cargo:warning=                                                                  ^~~~~~~~~~~~~~~~
cargo:warning=                                                                  NFNL_MSG_BATCH_END
cargo:warning=cc1: error: unrecognized command line option '-Wno-unknown-warning-option' [-Werror]
cargo:warning=cc1: error: unrecognized command line option '-Wno-address-of-packed-member' [-Werror]
cargo:warning=cc1: all warnings being treated as errors

It seems these constants are not available in the musl dockers. However, I don't think the existence of these constants depends on the musl vs glibc platform. These come from kernel headers and I think the headers in these specific containers are simply older than the constants. Furthermore, as the code that handles them is inside kernel, not musl or other userspace library, even the existence or non-existence of the constant in some C header doesn't say much about the constant being actually useful ‒ one could compile the program on a system without the constant defined, but if there was a new enough kernel running or one took the program to other (newer) system, it would work fine.

Therefore, I don't think the right way is to somehow conditionally define or not define the constants on the rust side. On the other hand, I myself don't need these specific constants, I've included them more for completeness sake. So, what's the right way forward?

I'm also getting some errors on the emacscripten platforms, which seems a bit odd ‒ I didn't touch these and even as I read the tests, the headers I've added there should not get used there. Am I missing something?

warning: cache:INFO: generating system asset: is_vanilla.txt... (this will be cached in "/emsdk-portable/.emscripten_cache/is_vanilla.txt" for subsequent builds)
warning: cache:INFO:  - ok
warning: shared:INFO: (Emscripten: Running sanity checks)
warning: shared:WARNING: java does not seem to exist, required for closure compiler, which is optional (define JAVA in /emsdk-portable/.emscripten if you want it)
warning: shared:WARNING: closure compiler will not be available
warning: src/cmsg.c:14:9: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
warning:         return CMSG_NXTHDR(msgh, cmsg);
warning:                ^~~~~~~~~~~~~~~~~~~~~~~
warning: /emsdk-portable/fastcomp/emscripten/system/include/libc/sys/socket.h:286:44: note: expanded from macro 'CMSG_NXTHDR'
warning:         __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
warning:         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: 1 warning generated.
     Running `CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_AUTHORS='Alex Crichton <alex@alexcrichton.com>' OUT_DIR=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out CARGO_PKG_NAME=libc-test CARGO_PKG_VERSION_MINOR=1 CARGO_MANIFEST_DIR=/checkout/libc-test CARGO_PKG_DESCRIPTION= CARGO_PKG_VERSION=0.1.0 LD_LIBRARY_PATH='/checkout/target/debug/deps:/rust/lib' CARGO=/rust/bin/cargo CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_PRE= CARGO_PKG_HOMEPAGE= CARGO_PKG_REPOSITORY= rustc --crate-name cmsg libc-test/test/cmsg.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C debuginfo=2 --test -C metadata=ee67ef54c3f4ef7d -C extra-filename=-ee67ef54c3f4ef7d --out-dir /checkout/target/asmjs-unknown-emscripten/debug/deps --target asmjs-unknown-emscripten -C incremental=/checkout/target/asmjs-unknown-emscripten/debug/incremental -L dependency=/checkout/target/asmjs-unknown-emscripten/debug/deps -L dependency=/checkout/target/debug/deps --extern libc=/checkout/target/asmjs-unknown-emscripten/debug/deps/liblibc-84bb7fda307a2e4f.rlib -L native=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out -L native=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out -l static=cmsg -l static=main`
     Running `CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_AUTHORS='Alex Crichton <alex@alexcrichton.com>' OUT_DIR=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out CARGO_PKG_NAME=libc-test CARGO_PKG_VERSION_MINOR=1 CARGO_MANIFEST_DIR=/checkout/libc-test CARGO_PKG_DESCRIPTION= CARGO_PKG_VERSION=0.1.0 LD_LIBRARY_PATH='/checkout/target/debug/deps:/rust/lib' CARGO=/rust/bin/cargo CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_PRE= CARGO_PKG_HOMEPAGE= CARGO_PKG_REPOSITORY= rustc --crate-name linux_ipv6 libc-test/test/linux_ipv6.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C debuginfo=2 --cfg test -C metadata=9af589d3f2be80e4 -C extra-filename=-9af589d3f2be80e4 --out-dir /checkout/target/asmjs-unknown-emscripten/debug/deps --target asmjs-unknown-emscripten -C incremental=/checkout/target/asmjs-unknown-emscripten/debug/incremental -L dependency=/checkout/target/asmjs-unknown-emscripten/debug/deps -L dependency=/checkout/target/debug/deps --extern libc=/checkout/target/asmjs-unknown-emscripten/debug/deps/liblibc-84bb7fda307a2e4f.rlib -L native=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out -L native=/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out -l static=cmsg -l static=main`
error: linking with `emcc` failed: exit code: 1
  |
  = note: "emcc" "-s" "DISABLE_EXCEPTION_CATCHING=1" "-L" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.1t6fxx84e0uoo8i9.rcgu.o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.2gi8w8u3iwftmocc.rcgu.o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.2zha6zu6ff4a349x.rcgu.o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.3kpye8c3pvre7xvh.rcgu.o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.3wxasxblo83k7hys.rcgu.o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.nbdj1tb40h0wz7a.rcgu.o" "-o" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.js" "-s" "EXPORTED_FUNCTIONS=[\"_main\",\"_rust_eh_personality\"]" "/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.k1k299hmqotk4a2.rcgu.o" "-O0" "--memory-init-file" "0" "-g4" "-s" "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=[]" "-L" "/checkout/target/asmjs-unknown-emscripten/debug/deps" "-L" "/checkout/target/debug/deps" "-L" "/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out" "-L" "/checkout/target/asmjs-unknown-emscripten/debug/build/libc-test-16056992861f7108/out" "-L" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib" "-l" "cmsg" "-l" "main" "/checkout/target/asmjs-unknown-emscripten/debug/deps/liblibc-84bb7fda307a2e4f.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libstd-ca282257223c61af.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libpanic_abort-69f84fa9e08d836d.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libhashbrown-1eb893e37178b5f9.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/librustc_std_workspace_alloc-7fe7da1a52690f92.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libbacktrace-7412f65cabdf5f0c.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/librustc_demangle-5cc7d9548aeac745.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libunwind-04b16f62c35672e5.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libcfg_if-21b82f698a5d2ed8.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/liblibc-41fc7ae57924c552.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/liballoc-32b643f39e73fe82.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/librustc_std_workspace_core-7c3b612b759d1821.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libcore-9aff0002d636d375.rlib" "/rust/lib/rustlib/asmjs-unknown-emscripten/lib/libcompiler_builtins-fa1b74e3f2d98dc6.rlib" "-l" "c" "-l" "c" "-s" "ERROR_ON_UNDEFINED_SYMBOLS=1" "-s" "ASSERTIONS=1" "-s" "DISABLE_EXCEPTION_CATCHING=1" "-s" "ABORTING_MALLOC=0" "-s" "WASM=0"
  = note: shared:ERROR: fastcomp is not compatible with wasm object files:/checkout/target/asmjs-unknown-emscripten/debug/deps/linux_ipv6-9af589d3f2be80e4.1t6fxx84e0uoo8i9.rcgu.o

@gnzlbg
Copy link
Contributor

gnzlbg commented Oct 28, 2019

It seems these constants are not available in the musl dockers. However, I don't think the existence of these constants depends on the musl vs glibc platform. These come from kernel headers and I think the headers in these specific containers are simply older than the constants.

Kind of. You can't use the kernel headers with any C library, because they might have conflict (e.g. if the kernel #define FOO ... and the C library also #define FOO ... stuff doesn't build. In fact, the Linux kernel headers can often not be used if you use musl. The main way to be able to use both is to use a different version of the kernel headers, e.g., there are "musl sanitized" kernel headers for different Linux kernel versions that you can use.

The "musl sanitized" kernel headers that libc is verified against in the Linux musl build jobs might be a bit older than the normal ones, and might not contain these constants. If you look at the Dockerfiles in the ci/ directory, you can check that out, and maybe update them to a newer version.

@vorner
Copy link
Contributor Author

vorner commented Oct 29, 2019

I've checked. The docker uses 4.4.2-2, the released version is 4.4.2-3. The constants are not available there :-(. It's not even in their master. So, what are the options?

  • I could try asking the upstream to update it, wait for release and then update here. But I guess this might take some time.
  • I could remove these constants from the pull request and wait until they are older to include them, maybe setting a reminder in half a year or something.
  • The test could be somehow disabled on musl. After all, there's little chance the value would be different.

Anything else?

@vorner
Copy link
Contributor Author

vorner commented Nov 17, 2019

I've deleted the constants at fault to move forward somehow.

However, I'm still getting CI errors on platforms that are unrelated to the changes and that I haven't touched.

@gnzlbg Do you have an idea what might be happening in there? Is it something I've missed?

@gnzlbg
Copy link
Contributor

gnzlbg commented Nov 19, 2019

@bors: r+

@bors
Copy link
Contributor

bors commented Nov 19, 2019

📌 Commit e7936ad has been approved by gnzlbg

@vorner
Copy link
Contributor Author

vorner commented Nov 19, 2019

The javascript platforms still fail :-(, no idea why. Bors probably can do nothing about that.

Besides, should I first squash that fixup, before merging?

bors added a commit that referenced this pull request Nov 19, 2019
Nfnetfilter log

This adds the constants from linux/netfilter/nfnetlink.h and nfnetlink_log.h. These are the files I need for jbaublitz/neli#48. After this gets in, I'd like to follow-up with the other nfnetlink_*.h files too, as I'd like to extend neli with further protocols in the future, but I want to do a smaller PR first to see if there are some things to tweak.

I've noticed similar netfilter constants are also in the android subfolder, therefore I'm adding them there too (I don't like the copy-pasting, but it seems the other ones are already copy-pasted). I assume the test will catch it if anything is different on that platform.
@bors
Copy link
Contributor

bors commented Nov 19, 2019

⌛ Testing commit e7936ad with merge 9c3d8be...

@bors
Copy link
Contributor

bors commented Nov 19, 2019

💔 Test failed - status-azure

@gnzlbg
Copy link
Contributor

gnzlbg commented Nov 20, 2019

@bors: retry

@bors
Copy link
Contributor

bors commented Nov 20, 2019

⌛ Testing commit e7936ad with merge 51e047c...

bors added a commit that referenced this pull request Nov 20, 2019
Nfnetfilter log

This adds the constants from linux/netfilter/nfnetlink.h and nfnetlink_log.h. These are the files I need for jbaublitz/neli#48. After this gets in, I'd like to follow-up with the other nfnetlink_*.h files too, as I'd like to extend neli with further protocols in the future, but I want to do a smaller PR first to see if there are some things to tweak.

I've noticed similar netfilter constants are also in the android subfolder, therefore I'm adding them there too (I don't like the copy-pasting, but it seems the other ones are already copy-pasted). I assume the test will catch it if anything is different on that platform.
@bors
Copy link
Contributor

bors commented Nov 20, 2019

☀️ Test successful - checks-cirrus-freebsd-10, checks-cirrus-freebsd-11, checks-cirrus-freebsd-12, status-azure
Approved by: gnzlbg
Pushing 51e047c to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants