Improve perf for String.filter up to 3x #9509

abelbraaksma · 2020-06-19T18:26:16Z

This PR is a follow-up on the findings in this comment in the main issue: #9390 (comment) (scroll to String.filter).

I've added the case for LOH scenarios to fallback to the old method, to prevent extra LOH allocations, however, this ultimately depends on the user-supplied filter, obviously. SB internally keeps an array and if that array exceeds the LOH threshold by repeated Appends there's little we can do. Also note that in general, SB will give more GC pressure, so there'll be a jump in the performance back to the old perf for strings > 40_000 chars (I didn't use the more exact threshold of 42_500 for having a safety margin, fcs uses this too with similar reasons).

For the average case, this gives a perf gain of 1.8x to ~3x depending on how much is filtered out. The rare case where input is large and filtering filters out everything, we still get a 35% gain.

Reason for not using Array.filter is two-fold: it is not in scope yet (and the code is complex) and it is slower than the cut-off method shown here when the array contains value types (here char).

For ease of reference, here's the chart (where None means: filter out all characters, and All means keep all of the input after filter). The proposed method is buildAndCut from the chart, modified for LOH:

KevinRansom

Please change the test string, otherwise I like this a lot.

src/fsharp/FSharp.Core/string.fs

tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Collections/StringModule.fs

abelbraaksma · 2020-06-20T10:49:12Z

@KevinRansom, done, resolved :)

src/fsharp/FSharp.Core/string.fs

abelbraaksma · 2020-06-24T20:15:43Z

@KevinRansom, your change requests were done, can you re-review (I resolved all, but it still says the change is requested... oddity of Github)?

KevinRansom

Thank you for this.

* Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (#9577) Microsoft.DotNet.Arcade.Sdk From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Improve perf for String.filter up to 3x (#9509) * Improve perf for String.filter 2-2.5x * Cleanup: remove "foo" etc in tests * Add tests for new execution path for LOH in String.filter * Change test string * String map performance improvement (#9470) * Simplify and improve perf of String.length * Improve performance of String.map * Revert "Simplify and improve perf of String.length" * Resolves #9470 (comment) * Lingering space * Change `String` to use `new` to clarify use of ctor * Add some better tests for String.map, add side-effect test * Add tests to ensure the mapping function is called a deterministically amount of times * Fix typo * Remove "foo" from String.map tests * Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (#9512) * Turn String.replicate from O(n) into O(log(n)) * Cleanup String.replicate tests by removing usages of "foo" * String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points * Improve String.replicate algorithm further * Add tests for String.replicate covering all lines/branches of algo * Fix accidental comment Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl>

* string interploation implementation * string interploation tests * escape {{ }}, test verbatim and triple quote, implement .NET specifiers * fix tests * string interpolation tests: internal representation corner cases * string-interp tests should have --langversion:preview * string interop tests: sprintf * string interp tests: format specifier negative cases * string interp tests: format specifier negative cases, .NET-style padding * fix nested interp strings * style cleanup * lex: unify string interp stack and counter * string-interp: add test cases * fix mixed quote nested string interpolation * string-interp: add test case for multiple interpolation points with different indentation * lexfilter: push new CtxtParen at endPos for INTERP_STRING_PART and INTERP_STRING_BEGIN_PART * lexfilter: do not check undentation limit for string interpolation tokens. * FormattableString prototype * add FormattableString support * negative error checking * remove diagnostics * simpler FormattableString implementation * fix test * add testing for nested * add IFormattable support * tweak error message * tests: StringInterpolation: fix case errors * fix error message * check number of values matches * allow use of format strings with printf and friends * update baselines * fix baselines * add Experimental attributes * update string interp negative tests * stringinterp test: add PrintFormat tests * printf: fix empty interpolation string evaluates to null in printf env * enable test corectly * Revert "printf: fix empty interpolation string evaluates to null in printf env" This reverts commit 7f39617. * simplify codegen for interpolated strings * fix build * fix build * Merge master to feature/string-interp (#9580) * Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (#9577) Microsoft.DotNet.Arcade.Sdk From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Improve perf for String.filter up to 3x (#9509) * Improve perf for String.filter 2-2.5x * Cleanup: remove "foo" etc in tests * Add tests for new execution path for LOH in String.filter * Change test string * String map performance improvement (#9470) * Simplify and improve perf of String.length * Improve performance of String.map * Revert "Simplify and improve perf of String.length" * Resolves #9470 (comment) * Lingering space * Change `String` to use `new` to clarify use of ctor * Add some better tests for String.map, add side-effect test * Add tests to ensure the mapping function is called a deterministically amount of times * Fix typo * Remove "foo" from String.map tests * Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (#9512) * Turn String.replicate from O(n) into O(log(n)) * Cleanup String.replicate tests by removing usages of "foo" * String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points * Improve String.replicate algorithm further * Add tests for String.replicate covering all lines/branches of algo * Fix accidental comment Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Re enable tests for operators: OperatorsModule1.fs and OperatorsModule2.fs (#9516) (#9589) * Re-enabling tests from OperatorsModule1/2.fs (compile errors) * Fix compile errors in OperatorsModule1/2.fs, fix tests. Note tanh test comment. * Fix `tanh` test, ensure stable result on x86 vs x64 runtimes * Stop using exception AssertionException, so that test window shows useful info * Whitespace cleanup and redundant code removal * Cleanup spelling etc * Re-enabling int, int16, int32, int64, nativeint, incr, nullArg etc tests * Special-case floating-point assertion messages for higher precision output * Fix/update/add tests (some still failing) * Separate Checked tests, add & fix others, differentiate framework/bitness for some tests * Add branch for .NET Native (ignore cos test) * Resorting to comparing floats with a delta using Assert.AreNearEqual * Add some more tests Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (#9576) (#9599) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (#9576) (#9604) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Merge master to feature/string-interp (#9615) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> * Merge master to feature/string-interp (#9619) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> * Text tweeks * don't auto-resolve types from System.Runtime.WindowsRuntime (#9644) (#9648) Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> * yeet (#9657) (#9661) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * yeet (#9657) (#9670) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * fix up tokenizer tests * fix code review things * fix code review things * fix code review things * fix code review things * add various testing * correct continuations for interpolated strings * fix lexer continuations and colorization for multi-line interpolated strings * revert xlf changes * fix assert * completion and brace matching (not all tests passing yet) * Fix rebuild * fix various niggles and get tests working * fix printf when '%a' in final position * fix test case * interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix build * fix missing error message * fix % specifiers for interpolated strings * fix % specifiers for interpolated strings * fix FCS tests * minor nits from code review * code review feedback and use struct tuples in more places * revert struct tuples * use struct tuples where possible, byrefs for index * fix byref for index * fix ksprintf block size * make recent cache entry more explicit (cleanup) * improve performance * remove unused code * Move existing Compiler.ComponentTests to a new Compiler.fs framework (#9839) (#9848) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com> * Move existing Compiler.ComponentTests to a new Compiler.fs framework (#9839) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages * fix #9893 * fix unmantched right brace in interp string Co-authored-by: Yatao Li <yatli@microsoft.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: dotnet bot <dotnet-bot@dotnetfoundation.org> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com>

* string interploation implementation * string interploation tests * escape {{ }}, test verbatim and triple quote, implement .NET specifiers * fix tests * string interpolation tests: internal representation corner cases * string-interp tests should have --langversion:preview * string interop tests: sprintf * string interp tests: format specifier negative cases * string interp tests: format specifier negative cases, .NET-style padding * fix nested interp strings * style cleanup * lex: unify string interp stack and counter * string-interp: add test cases * fix mixed quote nested string interpolation * string-interp: add test case for multiple interpolation points with different indentation * lexfilter: push new CtxtParen at endPos for INTERP_STRING_PART and INTERP_STRING_BEGIN_PART * lexfilter: do not check undentation limit for string interpolation tokens. * FormattableString prototype * add FormattableString support * negative error checking * remove diagnostics * simpler FormattableString implementation * fix test * add testing for nested * add IFormattable support * tweak error message * tests: StringInterpolation: fix case errors * fix error message * check number of values matches * allow use of format strings with printf and friends * update baselines * fix baselines * add Experimental attributes * update string interp negative tests * stringinterp test: add PrintFormat tests * printf: fix empty interpolation string evaluates to null in printf env * enable test corectly * Revert "printf: fix empty interpolation string evaluates to null in printf env" This reverts commit 7f39617. * simplify codegen for interpolated strings * fix build * fix build * Merge master to feature/string-interp (dotnet#9580) * Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (dotnet#9577) Microsoft.DotNet.Arcade.Sdk From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Improve perf for String.filter up to 3x (dotnet#9509) * Improve perf for String.filter 2-2.5x * Cleanup: remove "foo" etc in tests * Add tests for new execution path for LOH in String.filter * Change test string * String map performance improvement (dotnet#9470) * Simplify and improve perf of String.length * Improve performance of String.map * Revert "Simplify and improve perf of String.length" * Resolves dotnet#9470 (comment) * Lingering space * Change `String` to use `new` to clarify use of ctor * Add some better tests for String.map, add side-effect test * Add tests to ensure the mapping function is called a deterministically amount of times * Fix typo * Remove "foo" from String.map tests * Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (dotnet#9512) * Turn String.replicate from O(n) into O(log(n)) * Cleanup String.replicate tests by removing usages of "foo" * String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points * Improve String.replicate algorithm further * Add tests for String.replicate covering all lines/branches of algo * Fix accidental comment Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Re enable tests for operators: OperatorsModule1.fs and OperatorsModule2.fs (dotnet#9516) (dotnet#9589) * Re-enabling tests from OperatorsModule1/2.fs (compile errors) * Fix compile errors in OperatorsModule1/2.fs, fix tests. Note tanh test comment. * Fix `tanh` test, ensure stable result on x86 vs x64 runtimes * Stop using exception AssertionException, so that test window shows useful info * Whitespace cleanup and redundant code removal * Cleanup spelling etc * Re-enabling int, int16, int32, int64, nativeint, incr, nullArg etc tests * Special-case floating-point assertion messages for higher precision output * Fix/update/add tests (some still failing) * Separate Checked tests, add & fix others, differentiate framework/bitness for some tests * Add branch for .NET Native (ignore cos test) * Resorting to comparing floats with a delta using Assert.AreNearEqual * Add some more tests Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9599) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9604) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Merge master to feature/string-interp (dotnet#9615) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (dotnet#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> * Merge master to feature/string-interp (dotnet#9619) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (dotnet#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> * Text tweeks * don't auto-resolve types from System.Runtime.WindowsRuntime (dotnet#9644) (dotnet#9648) Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> * yeet (dotnet#9657) (dotnet#9661) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * yeet (dotnet#9657) (dotnet#9670) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * fix up tokenizer tests * fix code review things * fix code review things * fix code review things * fix code review things * add various testing * correct continuations for interpolated strings * fix lexer continuations and colorization for multi-line interpolated strings * revert xlf changes * fix assert * completion and brace matching (not all tests passing yet) * Fix rebuild * fix various niggles and get tests working * fix printf when '%a' in final position * fix test case * interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix build * fix missing error message * fix % specifiers for interpolated strings * fix % specifiers for interpolated strings * fix FCS tests * minor nits from code review * code review feedback and use struct tuples in more places * revert struct tuples * use struct tuples where possible, byrefs for index * fix byref for index * fix ksprintf block size * make recent cache entry more explicit (cleanup) * improve performance * remove unused code * Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839) (dotnet#9848) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com> * Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages * fix dotnet#9893 * fix unmantched right brace in interp string Co-authored-by: Yatao Li <yatli@microsoft.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: dotnet bot <dotnet-bot@dotnetfoundation.org> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com>

* Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (dotnet#9577) Microsoft.DotNet.Arcade.Sdk From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Improve perf for String.filter up to 3x (dotnet#9509) * Improve perf for String.filter 2-2.5x * Cleanup: remove "foo" etc in tests * Add tests for new execution path for LOH in String.filter * Change test string * String map performance improvement (dotnet#9470) * Simplify and improve perf of String.length * Improve performance of String.map * Revert "Simplify and improve perf of String.length" * Resolves dotnet#9470 (comment) * Lingering space * Change `String` to use `new` to clarify use of ctor * Add some better tests for String.map, add side-effect test * Add tests to ensure the mapping function is called a deterministically amount of times * Fix typo * Remove "foo" from String.map tests * Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (dotnet#9512) * Turn String.replicate from O(n) into O(log(n)) * Cleanup String.replicate tests by removing usages of "foo" * String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points * Improve String.replicate algorithm further * Add tests for String.replicate covering all lines/branches of algo * Fix accidental comment Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl>

* string interploation implementation * string interploation tests * escape {{ }}, test verbatim and triple quote, implement .NET specifiers * fix tests * string interpolation tests: internal representation corner cases * string-interp tests should have --langversion:preview * string interop tests: sprintf * string interp tests: format specifier negative cases * string interp tests: format specifier negative cases, .NET-style padding * fix nested interp strings * style cleanup * lex: unify string interp stack and counter * string-interp: add test cases * fix mixed quote nested string interpolation * string-interp: add test case for multiple interpolation points with different indentation * lexfilter: push new CtxtParen at endPos for INTERP_STRING_PART and INTERP_STRING_BEGIN_PART * lexfilter: do not check undentation limit for string interpolation tokens. * FormattableString prototype * add FormattableString support * negative error checking * remove diagnostics * simpler FormattableString implementation * fix test * add testing for nested * add IFormattable support * tweak error message * tests: StringInterpolation: fix case errors * fix error message * check number of values matches * allow use of format strings with printf and friends * update baselines * fix baselines * add Experimental attributes * update string interp negative tests * stringinterp test: add PrintFormat tests * printf: fix empty interpolation string evaluates to null in printf env * enable test corectly * Revert "printf: fix empty interpolation string evaluates to null in printf env" This reverts commit 7f39617. * simplify codegen for interpolated strings * fix build * fix build * Merge master to feature/string-interp (dotnet#9580) * Update dependencies from https://github.com/dotnet/arcade build 20200626.2 (dotnet#9577) Microsoft.DotNet.Arcade.Sdk From Version 1.0.0-beta.20302.3 -> To Version 1.0.0-beta.20326.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Improve perf for String.filter up to 3x (dotnet#9509) * Improve perf for String.filter 2-2.5x * Cleanup: remove "foo" etc in tests * Add tests for new execution path for LOH in String.filter * Change test string * String map performance improvement (dotnet#9470) * Simplify and improve perf of String.length * Improve performance of String.map * Revert "Simplify and improve perf of String.length" * Resolves dotnet#9470 (comment) * Lingering space * Change `String` to use `new` to clarify use of ctor * Add some better tests for String.map, add side-effect test * Add tests to ensure the mapping function is called a deterministically amount of times * Fix typo * Remove "foo" from String.map tests * Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement (dotnet#9512) * Turn String.replicate from O(n) into O(log(n)) * Cleanup String.replicate tests by removing usages of "foo" * String.replicate: add tests for missing cases, and for the new O(log(n)) cut-off points * Improve String.replicate algorithm further * Add tests for String.replicate covering all lines/branches of algo * Fix accidental comment Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Re enable tests for operators: OperatorsModule1.fs and OperatorsModule2.fs (dotnet#9516) (dotnet#9589) * Re-enabling tests from OperatorsModule1/2.fs (compile errors) * Fix compile errors in OperatorsModule1/2.fs, fix tests. Note tanh test comment. * Fix `tanh` test, ensure stable result on x86 vs x64 runtimes * Stop using exception AssertionException, so that test window shows useful info * Whitespace cleanup and redundant code removal * Cleanup spelling etc * Re-enabling int, int16, int32, int64, nativeint, incr, nullArg etc tests * Special-case floating-point assertion messages for higher precision output * Fix/update/add tests (some still failing) * Separate Checked tests, add & fix others, differentiate framework/bitness for some tests * Add branch for .NET Native (ignore cos test) * Resorting to comparing floats with a delta using Assert.AreNearEqual * Add some more tests Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9599) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) (dotnet#9604) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> * Merge master to feature/string-interp (dotnet#9615) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (dotnet#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> * Merge master to feature/string-interp (dotnet#9619) * Moved fsharpqa/Libraries/Core/Unchecked test cases to NUnit (dotnet#9576) * Moved fsharpqa/Libraries/Core/Reflectiontest cases to NUnit (dotnet#9611) * Migrated PreComputedTupleConstructor01.fs test case * Migrated PreComputedTupleConstructor02.fs test case * Migrated DU.fs and Record.fs test cases * Allow notebook to discover location of shared framework (dotnet#9596) Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> * Text tweeks * don't auto-resolve types from System.Runtime.WindowsRuntime (dotnet#9644) (dotnet#9648) Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> * yeet (dotnet#9657) (dotnet#9661) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * yeet (dotnet#9657) (dotnet#9670) yeet Co-authored-by: Phillip Carter <pcarter@fastmail.com> * fix up tokenizer tests * fix code review things * fix code review things * fix code review things * fix code review things * add various testing * correct continuations for interpolated strings * fix lexer continuations and colorization for multi-line interpolated strings * revert xlf changes * fix assert * completion and brace matching (not all tests passing yet) * Fix rebuild * fix various niggles and get tests working * fix printf when '%a' in final position * fix test case * interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix triple quote interpolated string specifer highlighting * fix build * fix missing error message * fix % specifiers for interpolated strings * fix % specifiers for interpolated strings * fix FCS tests * minor nits from code review * code review feedback and use struct tuples in more places * revert struct tuples * use struct tuples where possible, byrefs for index * fix byref for index * fix ksprintf block size * make recent cache entry more explicit (cleanup) * improve performance * remove unused code * Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839) (dotnet#9848) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com> * Move existing Compiler.ComponentTests to a new Compiler.fs framework (dotnet#9839) * Move existing Compiler.ComponentTests to a new Compiler.fs framework; Add 'parse' function * Changed some wording in error messages * fix dotnet#9893 * fix unmantched right brace in interp string Co-authored-by: Yatao Li <yatli@microsoft.com> Co-authored-by: Kevin Ransom (msft) <codecutter@hotmail.com> Co-authored-by: dotnet bot <dotnet-bot@dotnetfoundation.org> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Abel Braaksma <abel.online@xs4all.nl> Co-authored-by: Thorsten Reichert <ThorstenReichert@users.noreply.github.com> Co-authored-by: Phillip Carter <pcarter@fastmail.com> Co-authored-by: Brett V. Forsgren <brettfo@microsoft.com> Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com>

Improve perf for String.filter 2-2.5x

2c0884b

abelbraaksma changed the title ~~Improve perf for String.filter 2-2.5x~~ Improve perf for String.filter up to 3x Jun 19, 2020

abelbraaksma added 2 commits June 20, 2020 00:16

Cleanup: remove "foo" etc in tests

1577e1e

Add tests for new execution path for LOH in String.filter

ca0fadd

KevinRansom requested changes Jun 20, 2020

View reviewed changes

src/fsharp/FSharp.Core/string.fs Show resolved Hide resolved

tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Collections/StringModule.fs Outdated Show resolved Hide resolved

Change test string

c7d4cda

Liminiens reviewed Jun 20, 2020

View reviewed changes

src/fsharp/FSharp.Core/string.fs Show resolved Hide resolved

abelbraaksma requested a review from KevinRansom June 24, 2020 20:15

KevinRansom approved these changes Jun 24, 2020

View reviewed changes

cartermp approved these changes Jun 27, 2020

View reviewed changes

cartermp merged commit 7b29d44 into dotnet:master Jun 27, 2020

abelbraaksma deleted the String.filter-perf branch June 27, 2020 16:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve perf for String.filter up to 3x #9509

Improve perf for String.filter up to 3x #9509

abelbraaksma commented Jun 19, 2020 •

edited

Loading

KevinRansom left a comment

abelbraaksma commented Jun 20, 2020

abelbraaksma commented Jun 24, 2020 •

edited

Loading

KevinRansom left a comment

Improve perf for String.filter up to 3x #9509

Improve perf for String.filter up to 3x #9509

Conversation

abelbraaksma commented Jun 19, 2020 • edited Loading

KevinRansom left a comment

Choose a reason for hiding this comment

abelbraaksma commented Jun 20, 2020

abelbraaksma commented Jun 24, 2020 • edited Loading

KevinRansom left a comment

Choose a reason for hiding this comment

abelbraaksma commented Jun 19, 2020 •

edited

Loading

abelbraaksma commented Jun 24, 2020 •

edited

Loading