Switch to a smaller slugifier.

[tech4him1]

Currently, we use the slug module for creating URL slugs from titles. The problem is that slug bundles an entire Unicode lookup table, and that gets bundled into our distributable. it currently accounts for 30.5% of our bundle size. This should probably be changed to something a little lighter weight.

Related to #328.

[tech4him1]

@biilmann suggested using urlize from lib/urlHelper:
https://github.com/netlify/netlify-cms/blob/04ae5051ceafc0c600eace5e88671b2766f81fe6/src/lib/urlHelper.js#L15-L20

If we use this we may need to add the url library as a WebPack polyfill, or just install the polyfill ourselves (url on NPM).

[tech4him1]

Another option: The slugify package is a port of slug without the Unicode lookup table.

[erquhart]

If urlize can indeed do the trick, we should use it.

[tech4him1]

Possible slugifier tests:
urlize.spec.js
#501 (comment)
#414 (comment)

[tech4him1]

So here is what WordPress does slugification:

1. • Accented letters get the accents removed: remove_accents() (we are currently doing this).
   • The other option is to URL-encode, or just remove the chars.
2. • All other unsafe characters (Japanese, Chinese) get URL-encoded. Newer browsers will convert them on the fly, so to the user they still appear as normal characters.
   • The other option here is just to remove the unsafe characters (we are currently doing this).

@erquhart What's your opinion on the best way to do this?

[erquhart]

Newer browsers will convert them on the fly, so to the user they still appear as normal characters.

Not sure when this conversion would occur - would the resulting filename have these characters encoded?

[tech4him1]

Not sure when this conversion would occur - would the resulting filename have these characters encoded?

@erquhart Yes, in almost every case they would have to.

[tech4him1]

Just testing -- both DatoCMS and Forestry strip unsafe chars from the slug.

[erquhart]

If it's par for the course and simpler, let's do it.

[tech4him1]

So we have three options:

1. Stay with the current slug package, just tell it not to worry about converting emojis (this would remove the hash table).
2. Switch to the slugify package, it is pretty much the same expect more maintained and without hash table.
3. Use the urlize function that Matt ported, use the remove-accents package on top.

[tech4him1]

@biilmann @erquhart I'm trying to understand why the urlize function seems to leave some reserved characters in the URL. The list of RFC3986 unreserved characters is \w-._~, but urlize uses \w./\_#+-. Obviously those need to be kept for the entire URLs, but not for slugification. Is there something that I am missing here?