Don't throw exception when encoding text as UTF-8 bytes fails

[phoerious]

The following line throws an exception if the input HTML string cannot be encoded as UTF-8:

selectolax/selectolax/parser.pyx

Line 37 in 386392c

bytes_html = html.encode('UTF-8')

When using Selectolax for parsing arbitrary web data, this condition can be triggered easily when the server reports the wrong encoding or multiple encodings are mixed, particularly in combination with UTF-7.

Take the byte string b'Roboto+Condensed', which (on its own) should have correctly been decoded as either ASCII or UTF-8. However, interpreted as UTF-7, the string contains an invalid shift sequence:

>>> b'Roboto+Condensed'.decode('utf-7')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "../encodings/utf_7.py", line 12, in decode
    return codecs.utf_7_decode(input, errors, True)
UnicodeDecodeError: 'utf7' codec can't decode bytes in position 6-15: unterminated shift sequence

Ignoring this error will produce a Unicode string which cannot be re-encoded as UTF-8 bytes without either replacing or ignoring invalid characters:

>>> b'Roboto+Condensed'.decode('utf-7', errors='ignore')
'Robotoઉ\udd7a笞'
>>> b'Roboto+Condensed'.decode('utf-7', errors='ignore').encode('utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udd7a' in position 7: surrogates not allowed

Obviously, the easy fix is to do the encoding step beforehand, pass the raw bytes to Selectolax and force it to use UTF-8 by setting detect_encoding=False, but that's a lot of stuff you have to figure out in order to get it right, because the error is not really intuitive and can occur without any apparent reason. Since Selectolax already has a decode_errors parameter that defaults to "ignore", I would suggest to apply it also to this encoding step.

selectolax/selectolax/parser.pyx

Line 26 in 386392c

def __init__(self, html, detect_encoding=True, use_meta_tags=True, decode_errors = 'ignore'):

[rushter]

That's a good suggestion. Would you like to make a PR?

[phoerious]

Sure, can do.