Skip to content

Commit

Permalink
feat(mangler): use characters in the order of their likely frequency
Browse files Browse the repository at this point in the history
  • Loading branch information
sapphi-red committed Jan 28, 2025
1 parent 0c4c739 commit d6180da
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 13 deletions.
26 changes: 25 additions & 1 deletion crates/oxc_mangler/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -424,8 +424,32 @@ fn is_keyword(s: &str) -> bool {
#[repr(C, align(64))]
struct Aligned64([u8; 64]);

/// The characters are in frequency order, so that the characters with higher frequency are used first.
///
/// This idea was inspired by nanoid. <https://github.com/ai/nanoid/blob/5.0.9/url-alphabet/index.js>
///
/// This list was generated by the following steps:
/// 1. Generate a source code with replacing all manglable variable names with `$` (assuming `$` is the least used character).
/// You can do this by passing the following `blank` function to the `generate_name` parameter of [Mangler::build_with_symbols_and_scopes_impl].
/// ```rust
/// fn blank(_: usize) -> InlineString<12> {
/// let mut str = InlineString::new();
/// unsafe { str.push_unchecked(b"$"[0]); }
/// str
/// }
/// ```
/// 2. Run the following command in `target/minifier/default` to check generate the list:
/// ```
/// find . -type f -exec cat {} + | `# concat all files in that directory` \
/// tr -d '\n' | fold -w1 | `# separate each characters in to each line` \
/// grep -E '[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ$_0123456789]' | `# filter the character` \
/// sort | uniq -c | `# count each characters` \
/// sort -nr | awk '{print $2}' | tr -d '\n' `# format output`
/// ```
/// The result I got is `etnriaoscludfpmhg_10vy2436b8x579SCwTEDOkAjMNPFILRzBVHUWGKqJYXZQ`.
/// 3. Add `$` at the end and then move all numbers to the end of the list.
const BASE54_CHARS: Aligned64 =
Aligned64(*b"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ$_0123456789");
Aligned64(*b"etnriaoscludfpmhg_vybxSCwTEDOkAjMNPFILRzBVHUWGKqJYXZQ$1024368579");

/// Get the shortest mangled name for a given n.
/// Code adapted from [terser](https://github.com/terser/terser/blob/8b966d687395ab493d2c6286cc9dd38650324c11/lib/scope.js#L1041-L1051)
Expand Down
24 changes: 12 additions & 12 deletions tasks/minsize/minsize.snap
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
| Oxc | ESBuild | Oxc | ESBuild |
Original | minified | minified | gzip | gzip | Fixture
-------------------------------------------------------------------------------------
72.14 kB | 23.57 kB | 23.70 kB | 8.55 kB | 8.54 kB | react.development.js
72.14 kB | 23.57 kB | 23.70 kB | 8.52 kB | 8.54 kB | react.development.js

173.90 kB | 59.68 kB | 59.82 kB | 19.25 kB | 19.33 kB | moment.js
173.90 kB | 59.68 kB | 59.82 kB | 19.20 kB | 19.33 kB | moment.js

287.63 kB | 89.52 kB | 90.07 kB | 31.07 kB | 31.95 kB | jquery.js
287.63 kB | 89.52 kB | 90.07 kB | 30.95 kB | 31.95 kB | jquery.js

342.15 kB | 117.69 kB | 118.14 kB | 43.66 kB | 44.37 kB | vue.js
342.15 kB | 117.69 kB | 118.14 kB | 43.55 kB | 44.37 kB | vue.js

544.10 kB | 71.49 kB | 72.48 kB | 25.92 kB | 26.20 kB | lodash.js
544.10 kB | 71.49 kB | 72.48 kB | 25.89 kB | 26.20 kB | lodash.js

555.77 kB | 271.48 kB | 270.13 kB | 88.45 kB | 90.80 kB | d3.js
555.77 kB | 271.48 kB | 270.13 kB | 88.38 kB | 90.80 kB | d3.js

1.01 MB | 457.63 kB | 458.89 kB | 123.79 kB | 126.71 kB | bundle.min.js
1.01 MB | 457.63 kB | 458.89 kB | 123.53 kB | 126.71 kB | bundle.min.js

1.25 MB | 650.59 kB | 646.76 kB | 161.49 kB | 163.73 kB | three.js
1.25 MB | 650.59 kB | 646.76 kB | 161.11 kB | 163.73 kB | three.js

2.14 MB | 718.83 kB | 724.14 kB | 162.40 kB | 181.07 kB | victory.js
2.14 MB | 718.83 kB | 724.14 kB | 162.15 kB | 181.07 kB | victory.js

3.20 MB | 1.01 MB | 1.01 MB | 325.18 kB | 331.56 kB | echarts.js
3.20 MB | 1.01 MB | 1.01 MB | 324.36 kB | 331.56 kB | echarts.js

6.69 MB | 2.30 MB | 2.31 MB | 469.99 kB | 488.28 kB | antd.js
6.69 MB | 2.30 MB | 2.31 MB | 469.42 kB | 488.28 kB | antd.js

10.95 MB | 3.37 MB | 3.49 MB | 866.64 kB | 915.50 kB | typescript.js
10.95 MB | 3.37 MB | 3.49 MB | 864.74 kB | 915.50 kB | typescript.js

0 comments on commit d6180da

Please sign in to comment.