-
Notifications
You must be signed in to change notification settings - Fork 928
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replaced std::string with std::string_view and removed excessive copies in cudf::io #17734
Conversation
Nice optimization. General question, many of the strings in the PR (except for paths etc) are very tiny (< 10 chars) which may not impact performance or space otherwise. Should we just leave them as is? |
only a few, most of them aren't tiny (<16 characters) in the general case, regardless, incurring extra allocations even from small strings (>16 && < 1'000) when they aren't needed isn't ideal either as they lead to memory fragmentation for long-running programs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved. But I still have a different opinion that std::string_view
should not be used in associative containers.
Looks like we have build errors from replacing strings with string_views at certain places! |
Looks like we are getting some runtime errors in libcudf with this PR as well |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like some python tests are now failing due to extra "
chars in some of the output cols
Co-authored-by: David Wendt <45795991+davidwendt@users.noreply.github.com>
Co-authored-by: David Wendt <45795991+davidwendt@users.noreply.github.com>
/merge |
Description
As part of the improvement effort discussed in #15907, this merge request removes some of the excessive
std::string
copies and usesstd::string_view
in place ofstd::string
when the lifetime semantics are clear.std::string
is only replaced in this MR in linear functions and constructors, but not in structs as there's no established ownership or lifetime semantics to guarantee thestring_view
s will not outlive their source.There were also some cases of excessive copies, i.e. consider:
In the above example, the string is likely to be allocated twice if a temporary/string-literal is used to construct "s": one for the temporary and one for the copy constructor for
str
The string is only allocated once in all scenarios.
This also applies to
std::vector
and is arguably worse as there's no small-vector-optimization (i.e.std::string
's small-string-optimization/SSO).Checklist