Specify encoding for files

[kouralex]

Whilst trying to run Skosify on Windows, I encountered issues similar to this.

The answer there was to simply declare the encoding when you open a file. Supporting this user case requires some small changes in the code as well as a new parameter to the program, I suppose.

[osma]

Can you please show the error and traceback?

[kouralex]

There is no traceback provided by the program, only the message for the raised exception:

CRITICAL: Parsing failed. Exception: 'charmap' codec can't decode byte 0x8f in position 1132: character maps to <undefined>

As expected, I managed to run the program on WSL, though.

[libraryjeans]

Having the same issue. Thanks for the heads up

[osma]

There was a similar issue in Annif and it was fixed by this PR. But in Annif the input files are in text-based formats, not RDF (mostly).

I wonder what file caused this. Normally Skosify only parses the configuration file (optional) and RDF files. Can you try to work out which file caused this and perhaps post a minimal example file that causes the error? The error message seems to come from this line and is shown when the input RDF file cannot be parsed. But the parsing itself is handled by rdflib and in my understanding it should do the right thing w.r.t. character encodings - but apparently this isn't always the case.

My wishlist (for @kouralex or @libraryjeans):

• an example input file that causes this problem
• the command line used to run skosify
• debug output of skosify (use -D option on command line to enable this)

[TommiRTVA]

Hello, I have the same problem. Python 3.8 installed with Powershell.
Followed these instructions.
https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-windows-10
Command:
skosify -i .\yso-paikat-skos.rdf -o test.rdf
CRITICAL: Parsing failed. Exception: 'charmap' codec can't decode byte 0x81 in position 35428: character maps to
Used this file.
https://finto.fi/rest/v1/yso-paikat/data?format=application/rdf%2Bxml
Goal is to create custom SKOS-files based on these Finto-files.

[osma]

Possible fix on this branch: https://github.com/NatLibFi/Skosify/tree/issue62-utf8-encoding
Can you test it @kouralex or @TommiRTVA or @libraryjeans ?

[TommiRTVA]

Looks like it worked, at least the file has been analyced. Good!

skosify -i .\yso-paikat-skos.rdf -o test.rdf
INFO: Don't know what to do with literal http://purl.org/dc/terms/modified
INFO: Don't know what to do with literal http://purl.org/dc/terms/created
INFO: Don't know what to do with relation http://rdaregistry.info/Elements/u/P60683
INFO: Don't know what to do with relation http://purl.org/iso25964/skos-thes#broaderPartitive
INFO: Don't know what to do with relation http://rdaregistry.info/Elements/u/P60686
INFO: Don't know what to do with relation http://purl.org/iso25964/skos-thes#narrowerPartitive
WARNING: Concept scheme has no label(s). Use --label option to set the concept scheme label.
INFO: Some concepts not reached in initial cycle detection. Re-checking for loose concepts.
WARNING: Redundant hierarchical relationship http://www.yso.fi/onto/yso/p507242 skos:broader http://www.yso.fi/onto/yso/p109172 found, but not eliminated because eliminate_redundancy is not set
WARNING: Redundant hierarchical relationship http://www.yso.fi/onto/yso/p105341 skos:broader http://www.yso.fi/onto/yso/p94399 found, but not eliminated because eliminate_redundancy is not set
WARNING: Redundant hierarchical relationship http://www.yso.fi/onto/yso/p94168 skos:broader http://www.yso.fi/onto/yso/p94347 found, but not eliminated because eliminate_redundancy is not set
WARNING: Redundant hierarchical relationship http://www.yso.fi/onto/yso/p507241 skos:broader http://www.yso.fi/onto/yso/p109172 found, but not eliminated because eliminate_redundancy is not set
WARNING: Redundant hierarchical relationship http://www.yso.fi/onto/yso/p226189 skos:broader http://www.yso.fi/onto/yso/p94402 found, but not eliminated because eliminate_redundancy is not set

[brunoalmeida81]

Hello, I'm having this problem on Windows 10 Pro with Python 3.8.5. On cmd, when I try to run the script on yso-skos.rdf (obtained from Finto.fi) I get the following error: CRITICAL: Parsing failed. Exception: 'charmap' codec can't decode byte 0x9d in position 14043: character maps to . The script works as intended on a Ubuntu installation through WSL.