Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Categorize some failing libcxx tests #5231

Merged
merged 2 commits into from
Jan 14, 2025
Merged

Categorize some failing libcxx tests #5231

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d93cbd1
Categorize some failing libcxx tests
muellerj2 Jan 12, 2025
dac8bf7
Update bug title.
StephanTLavavej Jan 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions tests/libcxx/expected_results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,20 @@ std/numerics/rand/rand.dist/rand.dist.uni/rand.dist.uni.real/param_ctor.pass.cpp
# LLVM-113609: [libc++][test] Non-rebindable test_alloc in string.capacity/deallocate_size.pass.cpp
std/strings/basic.string/string.capacity/deallocate_size.pass.cpp FAIL

# LLVM-116054: [libc++] vprint_[non]unicode(ostream &, string_view, Args&&...) should not pad the output
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/print.pass.cpp FAIL
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/vprint_nonunicode.pass.cpp FAIL
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/vprint_unicode.pass.cpp FAIL

# LLVM-119174: [libcxx][test] Silence nodiscard warnings for std::expected
std/utilities/expected/expected.expected/monadic/and_then.pass.cpp FAIL
std/utilities/expected/expected.expected/monadic/or_else.pass.cpp FAIL
std/utilities/expected/expected.expected/monadic/transform.pass.cpp FAIL
std/utilities/expected/expected.expected/monadic/transform_error.pass.cpp FAIL

# LLVM-122638: [libc++][test] re.regex.construct/bad_backref.cpp assumes non-standard extension to extended regular expressions
StephanTLavavej marked this conversation as resolved.
Show resolved Hide resolved
std/re/re.regex/re.regex.construct/bad_backref.pass.cpp FAIL

StephanTLavavej marked this conversation as resolved.
Show resolved Hide resolved
# Non-Standard regex behavior.
# "It seems likely that the test is still non-conforming due to how libc++ handles the 'w' character class."
std/re/re.traits/lookup_classname.pass.cpp FAIL
Expand Down Expand Up @@ -151,6 +159,9 @@ std/language.support/support.limits/support.limits.general/ranges.version.compil
# libc++ is missing various <format> DRs
std/language.support/support.limits/support.limits.general/format.version.compile.pass.cpp FAIL

# libc++ doesn't implement LWG-2584 (LLVM-99976)
std/re/re.regex/re.regex.construct/bad_escape.pass.cpp FAIL

# libc++ doesn't implement LWG-3670
std/ranges/range.factories/range.iota.view/iterator/member_typedefs.compile.pass.cpp FAIL

Expand Down Expand Up @@ -240,8 +251,10 @@ std/utilities/function.objects/func.wrap/func.wrap.func/func.wrap.func.con/ctad.

# *** MISSING LWG ISSUE RESOLUTIONS ***
# LWG-2503 "multiline option should be added to syntax_option_type"
std/re/re.alg/re.alg.search/no_update_pos.pass.cpp FAIL
std/re/re.const/re.matchflag/match_multiline.pass.cpp FAIL
std/re/re.const/re.matchflag/match_not_eol.pass.cpp FAIL
std/re/re.const/re.synopt/syntax_option_type.pass.cpp FAIL

# LWG-2532 "Satisfying a promise at thread exit" (Open)
# WCFB02 implements the proposed resolution for this issue
Expand Down Expand Up @@ -552,6 +565,12 @@ std/utilities/format/format.formatter/format.formatter.locking/enable_nonlocking
std/containers/sequences/vector.bool/shrink_to_fit.pass.cpp FAIL
std/containers/sequences/vector/vector.capacity/shrink_to_fit.pass.cpp FAIL

# We disagree on the syntax flags set by the default basic_regex constructor: we set 0, libc++ sets ECMAScript.
# Relates to LWG-3604.
std/re/re.regex/re.regex.construct/default.pass.cpp FAIL
std/re/re.regex/re.regex.nonmemb/re.regex.nmswap/swap.pass.cpp FAIL
std/re/re.regex/re.regex.swap/swap.pass.cpp FAIL

Comment on lines +568 to +573
Copy link
Contributor Author

@muellerj2 muellerj2 Jan 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure where to put this one. The underlying issue is that [re.regex.construct]/1 does not specify the value returned by flags() as a postcondition on the default constructor, so MSVC STL returns 0 (which could be considered more in line with the postconditions on all the other constructors that require that flags() must return the flags passed to the constructor unchanged) while libc++ returns regex_constants::ECMAScript (which is probably informed by [re.synopt]/1's statement that the default grammar is ECMAScript if no other grammar is requested).

LWG-3604 is related, but discusses the postcondition on the flags imposed by all the other constructors and argues that they are too strict. Here, the problem is kind of the opposite: The standard doesn't state a postcondition on the flags for the default constructor at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LWG-3604 does discuss this problem at the end:

Also, the constructors say "Postconditions: flags() returns f." This prevents an implementation from storing f|ECMAScript in flags() if no grammar element is present in f. This seems like an unnecessary restriction, and forces implementations to do extra work to check if the ECMAScript grammar is in use. Arguably, it would even be better to require implementations to set ECMAScript in flags() if no grammar element was set in the flags passed to the constructor. This problem was introduced by LWG 2330(i).

I agree that there's no obvious place for this one - "LIKELY BOGUS TESTS" seems as good as anything, since if the Standard is vague on what happens, the test shouldn't be expecting something specific. Good enough for me, and a vast improvement over the status quo of "not analyzed".


# *** LIKELY STL BUGS ***
# Not analyzed, likely STL bugs. Various assertions.
Expand All @@ -563,13 +582,6 @@ std/re/re.alg/re.alg.search/awk.pass.cpp FAIL
std/re/re.alg/re.alg.search/basic.pass.cpp FAIL
std/re/re.alg/re.alg.search/ecma.pass.cpp FAIL
std/re/re.alg/re.alg.search/extended.pass.cpp FAIL
std/re/re.alg/re.alg.search/no_update_pos.pass.cpp FAIL
std/re/re.const/re.synopt/syntax_option_type.pass.cpp FAIL
std/re/re.regex/re.regex.construct/bad_backref.pass.cpp FAIL
std/re/re.regex/re.regex.construct/bad_escape.pass.cpp FAIL
std/re/re.regex/re.regex.construct/default.pass.cpp FAIL
std/re/re.regex/re.regex.nonmemb/re.regex.nmswap/swap.pass.cpp FAIL
std/re/re.regex/re.regex.swap/swap.pass.cpp FAIL
std/re/re.traits/lookup_collatename.pass.cpp FAIL
std/re/re.traits/transform_primary.pass.cpp FAIL

Expand Down Expand Up @@ -1112,11 +1124,6 @@ std/input.output/filesystems/fs.op.funcs/fs.op.weakly_canonical/weakly_canonical
# Clang error: unknown attribute 'no_unique_address' ignored [-Werror,-Wunknown-attributes]
std/ranges/ranges_robust_against_no_unique_address.pass.cpp FAIL

# Not analyzed. Runtime assertion, we appear to be ignoring stream width.
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/print.pass.cpp FAIL
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/vprint_nonunicode.pass.cpp FAIL
std/input.output/iostream.format/output.streams/ostream.formatted/ostream.formatted.print/vprint_unicode.pass.cpp FAIL

# Not analyzed. test_chrono_leap_second.h wants us to provide a vendor-specific version of
# test_leap_second_create(), but it's unclear how to adapt their parameters to our implementation.
std/time/time.zone/time.zone.leap/assign.copy.pass.cpp FAIL
Expand Down