MonteCarloGenerate: Add more metadata to MonteCarlo_Meta_data_output

[ddj116]

Originally requested by a user in discussions in Ramtares GitLab merge request 6053, this MR implements a better and more robust solution than the originally proposed approach.

• Add type, dispersion, min_value, max_value and other relevant internal members of MonteCarloVariable* classes to the output of MonteCarlo_Meta_data_output
• Fix an edge case in MonteCarloVariableRandomStringSet::generate_assignment() if no string values are added to the instance. Add a verif sim run to cover this case.

[coveralls]

coverage: 55.85% (+0.1%) from 55.744%
when pulling bbf48ba on 1574-more-mc-metadata
into 9ead0a2 on master.

[ddj116]

FYI @sharmeye @hchen99 @jmpenn I believe this is ready for review. I've got some intermediate commits on the branch, one of which was producing a segfault in test/SIM_mc_generation, but it was so finnicky the segfault would disappear when building Trick with -g -O0, putting me in a chicken/egg situation where I couldn't debug the problem unless I removed the symbols which let me inspect the problem in the first place 🐔 🥚 . The tip of the branch is stable in both debug and non-debug builds of Trick. Suggest squashing commits when y'all are ready. Let me know if you want to discuss the details of the mysterious disappearing segfault.

[hchen99]

FYI @sharmeye @hchen99 @jmpenn I believe this is ready for review. I've got some intermediate commits on the branch, one of which was producing a segfault in test/SIM_mc_generation, but it was so finnicky the segfault would disappear when building Trick with -g -O0, putting me in a chicken/egg situation where I couldn't debug the problem unless I removed the symbols which let me inspect the problem in the first place 🐔 🥚 . The tip of the branch is stable in both debug and non-debug builds of Trick. Suggest squashing commits when y'all are ready. Let me know if you want to discuss the details of the mysterious disappearing segfault.

Thanks, Dan! We'll review.

[drcurry2]

Was there a reason you ended the line printout with a comma? The other summarize_variable functions don't have the comma at the end

[ddj116]

I think the comma should be removed here, then prepended to all derived class methods instead. In it's current state it results in an extra comma at the end if summarize_variable isn't called polymorphically:

test.x_file_lookup[2]: type=Prescribed, max_skip=0, is_dependent=1, filename=Modified_data/datafile.txt, column_number=1, first_column_number=1
test.x_fixed_value_double: type=Constant,

I'll take a look at making that change here soon.

Thanks @drcurry2! By the way - am I correct to assume the key=<value>, key=<value> format I've implemented here will work for your post processing tool use case? It's a slightly different format than your implementation but all the info is still there.

[drcurry2]

The format works, I just need to make minor edits to the veras processing to read the file with your changes. Already testing it out

[drcurry2]

@ddj116 Are you still going to make the changes you mentioned above ^

[ddj116]

Yes I believe I made those fixes already. Looks like the branch just merged so it's in Trick now. We'll have to update our reference to the Trick submodule before this goes live in ramtares. Prefer to wait for the next release but we can supersede that if needed, ping me if you still need this capability and it hasn't yet entered ramtares production branches.

[ddj116]

@hchen99 @jmpenn @sharmeye I can't make any sense of the MacOS failure to build Trick - something about errors coming from swig_int_templates.hh . A few CI pipelines back this same code (with a few minor differences) built fine on Mac. Could this be an non-determinism CI issue or do you think it's a real problem we need to track down?

[hchen99]

@hchen99 @jmpenn @sharmeye I can't make any sense of the MacOS failure to build Trick - something about errors coming from swig_int_templates.hh . A few CI pipelines back this same code (with a few minor differences) built fine on Mac. Could this be an non-determinism CI issue or do you think it's a real problem we need to track down?

I also checked out this branch and having no problem building trick and all tests on my Mac. I am gonna to give it a try in a container to see if anything needs updating for Mac workflow. Btw, thank y'all for addressing review comments.

[hchen99]

@hchen99 @jmpenn @sharmeye I can't make any sense of the MacOS failure to build Trick - something about errors coming from swig_int_templates.hh . A few CI pipelines back this same code (with a few minor differences) built fine on Mac. Could this be an non-determinism CI issue or do you think it's a real problem we need to track down?

I also checked out this branch and having no problem building trick and all tests on my Mac. I am gonna to give it a try in a container to see if anything needs updating for Mac workflow. Btw, thank y'all for addressing review comments.

Ok, the latest github macos 12 runner image had an update recently that had python updated from 3.11 to 3.12. And looks like Python 3.12 has following legacy Unicode APIs removed and swig_int_templates.hh uses PyUnicode_GET_SIZE function that has been removed thus the error.

• PyUnicode_WCHAR_KIND
• PyUnicode_AS_UNICODE()
• PyUnicode_AsUnicode()
• PyUnicode_AsUnicodeAndSize()
• PyUnicode_AS_DATA()
• PyUnicode_FromUnicode()
• PyUnicode_GET_SIZE()
• PyUnicode_GetSize()
• PyUnicode_GET_DATA_SIZE()

[sharmeye]

@ddj116 if you merge the latest master commit into this branch the MacOS test should pass. Go ahead and merge in the latest master commit and if it passes we'll merge this into the master branch.

[ddj116]

I merged with origin/master and squashed my commits with no change to code, and now I'm getting a different set of weird failures on RHEL 8 that seem to have nothing to due with my code:

/usr/bin/g++ -g -I/usr/include -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -I/usr/include/udunits2 -std=c++14 -DLIBCLANG_MAJOR=16 -DLIBCLANG_MINOR=0 -DLIBCLANG_PATCHLEVEL=6 -DTRICK_GCC_VERSION=\"8.5.0\" -c PrintFileContents10.cpp -o object_Linux_8.5_x86_64/PrintFileContents10.o
In file included from /usr/include/clang/Basic/LLVM.h:21,
                 from /usr/include/clang/AST/APValue.h:16,
                 from /usr/include/clang/AST/Decl.h:16,
                 from ConstructValues.hh:9,
                 from EnumValues.hh:9,
                 from EnumValues.cpp:4:
/usr/include/llvm/Support/Casting.h:266:32: error: 'optional' is not a member of 'std'

Comparing the same build line to the last successful set of jobs, there's a difference in the value of -DLIBCLANG_PATCHLEVEL. Looks like it's 7 for the success on origin/master but 6 for this PR. Any ideas on what controls this value or if this is the root issue?

[hchen99]

I merged with origin/master and squashed my commits with no change to code, and now I'm getting a different set of weird failures on RHEL 8 that seem to have nothing to due with my code:

/usr/bin/g++ -g -I/usr/include -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -I/usr/include/udunits2 -std=c++14 -DLIBCLANG_MAJOR=16 -DLIBCLANG_MINOR=0 -DLIBCLANG_PATCHLEVEL=6 -DTRICK_GCC_VERSION=\"8.5.0\" -c PrintFileContents10.cpp -o object_Linux_8.5_x86_64/PrintFileContents10.o
In file included from /usr/include/clang/Basic/LLVM.h:21,
                 from /usr/include/clang/AST/APValue.h:16,
                 from /usr/include/clang/AST/Decl.h:16,
                 from ConstructValues.hh:9,
                 from EnumValues.hh:9,
                 from EnumValues.cpp:4:
/usr/include/llvm/Support/Casting.h:266:32: error: 'optional' is not a member of 'std'

Comparing the same build line to the last successful set of jobs, there's a difference in the value of -DLIBCLANG_PATCHLEVEL. Looks like it's 7 for the success on origin/master but 6 for this PR. Any ideas on what controls this value or if this is the root issue?

Probably more related to the the -DLIBCLANG_MAJOR. The last success build was using -DLIBCLANG_MAJOR=15 and the failed one using -DLIBCLANG_MAJOR=16. Users have had issues with recent RHEL 8 update with llvm16. Trick llvm16 support is in work.
Although both ubuntu:22.04 images used (for success and fail builds) have Clang: 13.0.1, 14.0.0, 15.0.7. Deleted this line as it is not applicable here. The fix is in master ;-)

[sharmeye]

@ddj116 try merging master in again now. Every time you try to get this pull brought in something like this happens. Are you sure this branch isn't cursed?