Simplify locale parser

[robertbastian]

This keeps track of slices instead of indices during parsing, and also simplifies the parser, which removes some checks that were done redundantly before, which should improve performance.

Benchmarks for baseline aff4632:

bench_langid_constr
  Instructions:                2411 (-21.23489%)
  L1 Accesses:                 3020 (-25.28451%)
  L2 Accesses:                    0 (-100.0000%)
  RAM Accesses:                  44 (-8.333333%)
  Estimated Cycles:            4560 (-20.23789%)

bench_langid_compare_components
  Instructions:                  17 (No change)
  L1 Accesses:                   20 (No change)
  L2 Accesses:                    2 (+100.0000%)
  RAM Accesses:                   0 (-100.0000%)
  Estimated Cycles:              30 (-50.00000%)

bench_langid_compare_components_str
  Instructions:                  17 (No change)
  L1 Accesses:                   21 (+5.000000%)
  L2 Accesses:                    1 (No change)
  RAM Accesses:                   0 (-100.0000%)
  Estimated Cycles:              26 (-56.66667%)

bench_langid_strict_cmp
  Instructions:                 993 (No change)
  L1 Accesses:                 1229 (+0.162999%)
  L2 Accesses:                    2 (-33.33333%)
  RAM Accesses:                  21 (-4.545455%)
  Estimated Cycles:            1974 (-1.888668%)

bench_langid_matching
  Instructions:                  17 (No change)
  L1 Accesses:                   21 (+10.52632%)
  L2 Accesses:                    1 (-50.00000%)
  RAM Accesses:                   0 (-100.0000%)
  Estimated Cycles:              26 (-59.37500%)

bench_langid_matching_str
  Instructions:                1437 (-21.85971%)
  L1 Accesses:                 1679 (-27.97083%)
  L2 Accesses:                    2 (No change)
  RAM Accesses:                  29 (-9.375000%)
  Estimated Cycles:            2704 (-21.87229%)

bench_langid_serialize
  Instructions:                4042 (No change)
  L1 Accesses:                 5609 (No change)
  L2 Accesses:                    1 (No change)
  RAM Accesses:                  37 (No change)
  Estimated Cycles:            6909 (No change)

bench_langid_serialize_writeable
  Instructions:                2657 (No change)
  L1 Accesses:                 3605 (+0.055509%)
  L2 Accesses:      18446744073709551615 (No change)
  RAM Accesses:                  36 (-5.263158%)
  Estimated Cycles:            4860 (-1.379870%)

bench_langid_canonicalize
  Instructions:                5502 (-10.56567%)
  L1 Accesses:                 7252 (-12.34135%)
  L2 Accesses:                    5 (No change)
  RAM Accesses:                  66 (-5.714286%)
  Estimated Cycles:            9587 (-10.80201%)

     Running benches/langid.rs (target/release/deps/langid-6ba4cda698d0abfb)
langid/overview         time:   [2.0350 µs 2.0362 µs 2.0375 µs]
                        change: [-18.305% -18.199% -18.092%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

langid/construct/langid time:   [590.86 ns 591.81 ns 593.34 ns]
                        change: [-30.688% -30.503% -30.278%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

langid/to_string/langid time:   [700.48 ns 701.12 ns 701.86 ns]
                        change: [-1.9481% -1.2349% -0.5345%] (p = 0.00 < 0.05)
                        Change within noise threshold.
langid/to_string/langid/writeable
                        time:   [430.48 ns 435.05 ns 441.35 ns]
                        change: [+0.4302% +0.9214% +1.6058%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe

langid/compare/struct/langid
                        time:   [71.456 ns 71.539 ns 71.629 ns]
                        change: [+1.1689% +1.2701% +1.3901%] (p = 0.00 < 0.05)
                        Performance has regressed.
langid/compare/str/langid
                        time:   [566.91 ns 569.34 ns 572.55 ns]
                        change: [-44.835% -44.335% -43.964%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  8 (8.00%) high severe
langid/compare/strict_cmp/langid
                        time:   [258.75 ns 258.98 ns 259.25 ns]
                        change: [-1.5186% -1.4039% -1.2925%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

langid/canonicalize/langid
                        time:   [1.6018 µs 1.6040 µs 1.6065 µs]
                        change: [-14.053% -13.612% -13.242%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

     Running benches/locale.rs (target/release/deps/locale-221b53c74f3bda80)
locale/overview         time:   [2.7926 µs 2.7946 µs 2.7968 µs]
                        change: [-26.881% -26.001% -25.333%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  5 (5.00%) high mild
  11 (11.00%) high severe

locale/construct/locale time:   [1.2332 µs 1.2338 µs 1.2344 µs]
                        change: [-40.525% -40.478% -40.431%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe

locale/to_string/locale time:   [864.57 ns 865.34 ns 866.19 ns]
                        change: [+3.2832% +3.4256% +3.5673%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
locale/to_string/locale/writeable
                        time:   [600.90 ns 601.50 ns 602.19 ns]
                        change: [+3.7332% +3.9913% +4.3575%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

locale/compare/struct/locale
                        time:   [133.01 ns 133.09 ns 133.17 ns]
                        change: [+2.8226% +3.4399% +4.0397%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe
locale/compare/str/locale
                        time:   [1.2465 µs 1.2480 µs 1.2497 µs]
                        change: [-35.046% -34.956% -34.867%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  1 (1.00%) high severe
locale/compare/strict_cmp/locale
                        time:   [432.02 ns 432.64 ns 433.35 ns]
                        change: [+0.3448% +0.5318% +0.7458%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

locale/canonicalize/locale
                        time:   [2.1397 µs 2.1421 µs 2.1448 µs]
                        change: [-29.123% -29.007% -28.897%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe

     Running benches/subtags.rs (target/release/deps/subtags-e201699efc1bec91)
subtags/language/parse  time:   [171.84 ns 171.97 ns 172.12 ns]
                        change: [-0.1141% -0.0249% +0.0704%] (p = 0.59 > 0.05)
                        No change in performance detected.

subtags/script/parse    time:   [53.062 ns 53.099 ns 53.138 ns]
                        change: [-6.0623% -5.9462% -5.8403%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

subtags/region/parse    time:   [138.64 ns 138.84 ns 139.03 ns]
                        change: [-0.2358% -0.0596% +0.1030%] (p = 0.49 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild

subtags/variant/parse   time:   [119.43 ns 119.70 ns 120.04 ns]
                        change: [+0.0043% +0.3204% +0.5969%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

[sffc]

I'd rather let @zbraniecki review this but if he doesn't get around to it I will take a look

[zbraniecki]

woa, this is cool! Thanks for digging into! I'm surprised by the perf win but happy to see it! The code is clean and the logic looks sound.