Support of UTF-8 in SPARQL Update queries

[KMax]

More information at rdflib-dev group -> https://groups.google.com/d/msg/rdflib-dev/fYiUeOLvMXM/k8VvcdkgwWgJ

Tested with OpenRDF Sesame 2.7.10.

[coveralls]

Coverage decreased (-2.04%) when pulling 0fe5885 on KMax:master into c429c4e on RDFLib:master.

[joernhees]

hmm, what encoding was used before hard wiring this to utf-8?
(i like utf-8, but i fear this could cause problems with other endpoints)

[gromgull]

I assume it was as god intended all strings to be - 7bit ascii!

[KMax]

Right, before that ascii was used (see the discussion in the group). Unfortunately, I don't have any other endpoint running to do testes, but I suppose utf-8 should be used everywhere.

Also, as understand Literal is a subclass of unicode datatype, so why the URL parameter was not encoded with utf-8.

[KMax]

Is there a way to write a test for these changes without usage of an external endpoint? Or some manual tests are enough to accept the changes?

[gromgull]

@KMax - we don't really have a good way to test it.

I was going to say you could test against the rdflib-web endpoint, but then remembered we still haven't implement UPDATE :)

Trying to read up on what the "correct" thing to do is, all I can find in the SPARQL protocol about encoding is this: http://www.w3.org/TR/sparql11-protocol/#query-via-post-direct

For direct POST, "Note that UTF-8 is the only valid charset here."

but this for directly having the update statements in the body, this is for form encoded querying.

I'll test it against fuseki, and if it works we'll merge, ok?

[gromgull]

So, I looked at Fuseki's web-interface, if posting an UPDATE statement with non-ascii unicode chars:

INSERT DATA { <urn:å> <urn:ü> <urn:ß> .  }

Chrome posts a plain content-type header (no charset) and this body:

update=INSERT+DATA+%7B+%3Curn%3A%C3%A5%3E+%3Curn%3A%C3%BC%3E+%3Curn%3A%C3%9F%3E+.++%7D

This is UTF-8, percent encoded.

[gromgull]

It works with with fuseki as well!

Thanks!

[joernhees]

we should fix those random test failures when endpoints are down...

[gromgull]

cough cough #269 #256 ...

[joernhees]

true... but see #364 nevertheless... it seems even though tests should be skipped on travis they aren't