Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix warnings in NCZarr tests #2816

Merged
merged 11 commits into from
Jan 24, 2024
Merged

Conversation

ZedThree
Copy link
Contributor

@ZedThree ZedThree commented Nov 30, 2023

Fixes #2768

Fixes the remaining 80 warnings in the NCZarr tests that aren't fixed by my other PRs.

I noticed that there's a Odometer struct along with some related free functions that are duplicated with some minor variations between various tests here and in nc4_tests -- potential for consolidating those into a single test helper library and reducing duplication?

`chunk_size` is only used if `PRINT_CHUNK_WASTE_REPORT` is
defined. Also move the declaration of `chunk_size` inside the `#ifdef`
to silence `set-but-unused` warning
`options->file` is allocated on the stack and so will never be `NULL`,
making this conditional always true. Instead, we want to check the
value of `file`. As `options` is allocated with `calloc`, we know that
`file` will be zero-initialised, and it's safe to just check the first
element
Copy link
Collaborator

@DennisHeimbigner DennisHeimbigner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config.h should always be first, so this:

#include <stddef.h>
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

should probably be this:

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef HAVE_STDDEF_H
#include <stddef.h>
#endif

@ZedThree
Copy link
Contributor Author

ZedThree commented Dec 1, 2023

Sure, I can make sure the config.h header is included first. The guard for stddef.h isn't really needed though, as it's a mandatory part of the C89 standard.

* main: (39 commits)
  Define USE_SZIP variable for nc-config.cmake.in
  matching cmake variables in autotools configuration
  moving the version into the project command in cmake
  Updated doxygen files for older, less-forgiving versions of doxygen (1.9.1, at least)
  Clean up doxygen warnings that were being treated as failures.
  Correcting a weird doxygen issue that has appeared.
  cmake: Fix Szip link using correct cmake var
  cmake: Improve FindSzip logic to provide a Szip_LIBRARY var
  Changed link to netCDF-Fortran documentation.
  Replaced ancient K&R function declarations to be C23 compatible
  Rebased PR by hand against main.
  count argument in H5Sselect_hyperslab
  Add H5FD_http_finalize function and call on hdf5 finalize
  Catching up on PRs, this is 2431 on the current 'main'
  Fix cmake syntax typo.
  Typo fix in support of gh2824.wif
  Fix typo in nc-config.cmake.in
  Removed a use of sprintf that required changing a function signature
  Replaced some sprintf with snprintf with aid of new variable containing size
  Replaced trivial uses of sprintf with snprintf
  ...
Copy link
Collaborator

@DennisHeimbigner DennisHeimbigner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some cases, converting from size_t to int is a problem
because on a 64 bit machine, sizeof(int)=4 and sizeof(size_t)=8.
For things that contain array indices (e.g. odometer) this can cause problems
because array indices can be larger than 2147483647. You would need to use
"long" instead of int.

@ZedThree
Copy link
Contributor Author

True, blindly changing size_t to int everywhere is probably not a good idea. Here, I've only changed places which currently generate warnings from implicit type conversions because either the sign could change or the value could change (larger type to small type, for instance), so if there was an issue, we should have seen it already (or the tests need expanding to catch it!)

If I've changed the type of a function argument or a struct member, it's because I can see everywhere it's used and what type is being passed.

So, for instance I've changed Odometer::rank from size_t to int because it's always used as an int (e.g. being passed to printvector), and always constructed from an int (e.g. from int Format::rank).

I do think it's really important to try and quash these warnings, as they can hide real issues and drown out static analysis. There's still almost 2000 warnings of this kind in main, and that's after I've fixed 500 of them. Just printing that many warnings measurably slows down compilation! I would really like to get to the point where this project could enable -Werror in CI. A lot of science depends on netCDF, I'd like it to be the best it could be!

@DennisHeimbigner
Copy link
Collaborator

I do think it's really important to try and quash these warnings,

Agreed. And it is probable that the main-line tests do not test constrained access
to very large datasets, hence odometer values > int max are not tested.
There is an option for running large tests that is normally not enabled.
It may test large datasets, I have not checked.
In any case, Odometer must be size_t not int. There are other possible instances of this
so you must be careful to analyze the domain of the size_t object.
Finding those places where the odometer is set with an int is important and those
places need to be fixed.

@WardF WardF added this to the 4.9.3 milestone Jan 17, 2024
@ZedThree
Copy link
Contributor Author

Not sure why just one test is failing, looks like it failed to build, but the output just says Error: The operation was canceled and I don't see where it's actually failing.

And it is probable that the main-line tests do not test constrained access
to very large datasets, hence odometer values > int max are not tested.

Just to be clear, this has only changed Odometer::rank and not Odometer::index, and rank only needs to be large enough to store values up to NC_MAX_VAR_DIMS == 1024

@WardF
Copy link
Member

WardF commented Jan 24, 2024

I think the failure was just one of those things that happens sometimes with Github Actions. I re-ran the test and it appears to have succeeded.

@WardF WardF merged commit 8add4ce into Unidata:main Jan 24, 2024
99 checks passed
@ZedThree ZedThree deleted the silence-nczarr-test-warnings branch March 14, 2024 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible bug in NcZarr tests
3 participants