Unicode support

[eslgastal]

Python's str.format() does not support unicode arguments (see http://bugs.python.org/issue7300). Thus doc2dash fails to build docs for e.g. https://github.com/JuliaLang/julia:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xd7' in position 0: ordinal not in range(128)

The fix is to use the % operator:

diff -ura doc2dash/parsers/intersphinx.py /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/intersphinx.py
--- doc2dash/parsers/intersphinx.py 2014-08-25 23:54:24.601090985 -0300
+++ /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/intersphinx.py  2014-08-25 23:30:39.885093298 -0300
@@ -80,7 +80,7 @@
     """
     link = soup.find('a', {'class': 'headerlink'}, href='#' + entry.anchor)
     tag = soup.new_tag('a')
-    tag['name'] = APPLE_REF_TEMPLATE.format(entry.type, entry.name)
+    tag['name'] = APPLE_REF_TEMPLATE % (entry.type, entry.name)
     if link:
         link.parent.insert(0, tag)
         return True
diff -ura doc2dash/parsers/pydoctor.py /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/pydoctor.py
--- doc2dash/parsers/pydoctor.py    2014-08-25 23:54:24.601090985 -0300
+++ /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/pydoctor.py 2014-08-25 23:30:49.149093284 -0300
@@ -57,7 +57,7 @@
         link = soup.find('a', attrs={'name': entry.anchor})
         if link:
             tag = soup.new_tag('a')
-            tag['name'] = APPLE_REF_TEMPLATE.format(entry.type, entry.name)
+            tag['name'] = APPLE_REF_TEMPLATE % (entry.type, entry.name)
             link.insert_before(tag)
             return True
         else:
diff -ura doc2dash/parsers/utils.py /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/utils.py
--- doc2dash/parsers/utils.py   2014-08-25 23:54:24.601090985 -0300
+++ /usr/local/lib/python2.7/dist-packages/doc2dash/parsers/utils.py    2014-08-25 23:30:29.613093316 -0300
@@ -84,7 +84,7 @@
         return start


-APPLE_REF_TEMPLATE = '//apple_ref/cpp/{}/{}'
+APPLE_REF_TEMPLATE = '//apple_ref/cpp/%s/%s'


 @coroutine

[hynek]

That would probably fix it although the underlying problem is that stuff should be unicode in the first place. Could you check whether #30 works to your satisfaction please? I was able to convert Julia’s documentation with 2.7, 3.4, and pypy and the results look good to me as far as I can tell.

[eslgastal]

#30 works for me! Thanks.

[hynek]

JFTR, 2.0.1 carrying this fix has been released just now.