Error loading geographical data with version 7.10.3211 #295
Comments
|
I have been able to recreate, even loading from This issue has been reported to development to look into ... |
|
Any progress on this one? I have following SPARQL failing (with another CRS than WGS84): Error is: I am running |
|
@yonyonson: This issue is still to be resolved, I have reported your occurrence to the bug report so it can be checked also ... |
|
any news on this issue? currently, we are trying to load the RDF dumps from DNB into Virtuoso ( |
|
Hi everyone, I've uploaded my Virtuoso instance to I have used this other structure: My question is if the previous structure is gonna be supported in next versions. |
|
@HughWilliams I also have the same issue, with the current |
|
@jakubklimek: GeoSPARQL support is being scheduled for the next Virtuoso 8 release ... |
|
My experience so far with Virtuoso OS 7.2.4 is that:
|
|
Hi, I am using OpenLink Virtuoso Server VOS So based on what I read from @ma-garcia, if I convert _:x <http://www.opengis.net/ont/geosparql#asWKT>
"<http://www.opengis.net/def/crs/OGC/1.3/CRS84>Point(-0.911101 41.635338)"^^<http://www.opengis.net/ont/geosparql#wktLiteral> .to _:x <http://www.opengis.net/ont/geosparql#crs>
<http://www.opengis.net/def/crs/OGC/1.3/CRS84>;
<http://www.opengis.net/ont/geosparql#asWKT>
"Point(-0.911101 41.635338)"^^<http://www.opengis.net/ont/geosparql#wktLiteral> .I should be able to load the data without an issue to Virtuoso, right? The other option, according to GH-773, it says that if a new datatype is used (and declare it as a Best Regards, |
|
Base GeoSPARQL support was added to the Virtuoso open source
-- and will soon be added to the Thus I would suggest building Virtuoso and the required Geospatial plugin from the |
|
@nandana where is this property coming from? By the way, I'm on Virtuoso |
|
I just took that from the previous example. Now I looked at the actual data I was loading from Wikidata, most of them are like the following and they are parsed without any error. But there are few other statements (related to the coordinates of Mars, I guess) like the following: They are not parsed correctly and they lead to the following error. I have not read the GeoSPARQL spec in detail but having a quick look at 8.5 Requirements for WKT Serialization (serialization=WKT) (page 34) and other pages, the above representation seems valid, isn't it? If not, I will have to go back to Wikidata devs :) In either case, I can also try with the develop/7 branch. |
|
@nandana - The described error is expected with VOS prior to the 7.2.6 update, or without the new plugin. Please let us know how things go with the latest |
|
Thanks @TallTed ! I've installed the latest version from develop/7 and following the guide to set GeoSPARQL. In the startup, it seems that those plugins are loaded correctly. but I still get the same error. Here's the content I am loading. Do you see any reason for the error? |
|
@nandana - It is often helpful to columnize your Turtle, as this can reveal oddness in the data that isn't so easy to see when the text is more tightly spaced. This appears to me to not be a valid I think that's meant to be -- I cannot tell where |
|
It looks odd to me too, but isn't such URIs are allowed in the WKT representation of geometry? For example, here it is says
or the following example from the GeoSPARQL spec Sec 8.5
This was taken directly from the Wikidata dump. If someone with GeoSPARQL expertise can confirm indeed this is not a valid value for |
|
@nandana - I see. You're correct, the GeoSPARQL spec does permit any valid URI to be used in this position (which seems silly, as it could lead to all sorts of nonsense data). I'm not sure why this data is not being accepted. @pkleef, @IvanMikhailov -- Any comment? That said, I note that Wikidata don't seem to have figured out how to address this, and their current kludge means only one CRS per celestial body (possibly including Earth; I didn't read deeply enough to be sure about this) in Wikidata, although multiple CRS are already used for locations on Mars, the Moon, and others (though perhaps not on Wikidata)... This is clearly an evolving space (pardon the pun). |
|
Thanks @TallTed ! One quick question related to Virtuoso + GeoSPARQL plugins. When VOS complains about a certain error, is there a way to get an idea about which line it is about (similar to the info given for syntax errors)? E.g.
These are files with ~24 million lines each and it is impossible to detect where the errors are using manual inspections. I could check for the previous type of errors using grep but this is something else. |
|
@nandana - I see your point about the error message, and have raised it internally to development. You may want to create an issue specific to that, so we can be sure to notify you when that enhancement is implemented. |
|
Thanks @TallTed ! I just created an issue for this. |
|
That wikidata Triple is IMHO not correct GeoSPARQL, since the URI is not an URI for a CRS. |
|
... and I'd personally avoid having this URI in the geometry serialization. Most geo-libraries won't read this as valid wkt |
|
When you try to load the following data using the latest version of virtuoso from develop/7: you receive the following error: The reason as correctly explained by @p1d1d1 is that this IRI does not represent a correct Coordinate Reference System (CRS) URI. Without a well defined coordinate system, these points are as useful as an amount without specifying the currency. |
|
@pkleef very nice implementation |
|
My reading of GeoSPARQL is that any URI may be used as a CRS URI within a This makes some sense, as all possible CRS do not yet exist, so future CRS can be supported without needing to update GeoSPARQL. I would think it better if there were a registry of CRS (similar to IANA MIME registry) and/or relationships between them (such that geodata expressed in "Mars 1979" can be related to "Mars 2000"), but this should not be necessary to load data in either/both/any CRS to an RDF store. The specific CRS URI should be evaluated only when queries need to compare geodata -- at which point such operations should lead to errors indicating that, for instance, "CRS (Yes, using a URI that does not refer to a CRS can lead to nonsense interpretation, just like "15 quatloos" cannot be usefully compared to "USD 15", but "15 quatloos" can be usefully compared to "30 quatloos" whenever that new cryptocurrency is created/defined.) |
|
@TallTed you're right, according to GeoSPARQL
But then it also says:
So URI identifying Mars are not ok for me. This lead as you say to nonsense data. |
|
@p1d1d1 Yes, I agree this might not be the best design decision from GeoSPARQL on how to represent CRS information in RDF. But I am more inclined towards what @TallTed said. Though they are not optimal, I think they are legal according to the current spec. I don't read the second paragraph you cited as OGC maintains an exclusive set of CRS URIs and nothing else is accepted. |
|
@p1d1d1 - As @nandana says, the spec says that the "OGC maintains a set of CRS URIs", it does not say the "OGC maintains the set of CRS URIs" — i.e., there is nothing that says that only URIs from that OGC list are valid CRS URIs. As things stand, the user is responsible for avoiding nonsense data, which may be painful for them, but this also means that they have the flexibility to adopt any appropriate CRS which may be developed in the future -- regardless of whether the OGC continues to maintain that list, or the OGC endorses/accepts the user's preferred new CRS, etc. As to whether this is the optimal solution... I think it's a similar conundrum to langtagged string literals. The CRS is necessary to interpret the Maybe there could be some syntactic sugar, such that "all Desktop GIS and other tools which read this data as invalid are just old -- so they want an older version of the data, which assumed that only one CRS did or ever would exist. Such tools will be updated to understand the new data which recognizes the reality of multiple CRS in active use, and old datasets which are based on any CRS other than the now-declared-default will be updated to include the CRS they're based on, and all will be well again. (Really, all will be well for the first time, as there have long been multiple CRS in use, and data sets which did not declare the CRS in use therein were effectively meaningless, and worse when used in combination with other data sets with undeclared — and frequently different — CRS.) |
|
@TallTed @p1d1d1 Long version: For example, converting the following to Nevertheless, now this data is interpreted against WGS84 and this is still failing as the coordinates are out of the bounds (i.e., |
|
@nandana — I am not immediately aware of a way to switch off these data checks; @IvanMikhailov or @pkleef may have a suggestion. That said — assuming that all the data you have to load is similar to the above, and that the CRS-including Thinking further, I might suggest you consider using some different dataset(s) and/or queries for your benchmarking, such as those developed by the LDBC, and written about in the old Virtuoso Blog among other places. |
|
I have ruun into this problem as well, and patched VOS as described in https://community.openlinksw.com/t/non-terrestrial-geo-literals/359 |
|
@pkleef, @IvanMikhailov, @kidehen, @openlink -- By not properly fixing this issue such that Virtuoso supports any URI as a valid CRS URI (as the spec requires, as discussed in detail above, particularly at 1 and 2), possibly among other fixes, we are leading/causing people like @asanchez75 to disable a significant chunk of data validation code, and @pfps to comment out a smaller selection, which may cause significant problems down the line -- not only by preferring non-OpenLink-branches of VOS. Virtuoso is a DBMS. Virtuoso should be managing the data people want to manage, not "protecting" people by refusing to manage data we don't like (i.e., geodata based on non-terrestrial or nonsensical or simply unknown CRS systems). |
* Added timeout to the sparql query * Fixed i18n issues in fct * Added support for IRIs from RDFviews * Added statistics about users of IRI as subject or object in graph to Metadata page * Commented out test code * Fixed use RDFa instead of microdata * Commented out debug lines * Fixed issues using current prefixes for DBpedia in demos * Fixed issues in b3s_get_types on weird data * Fixed issues with /describe page behind a (ssl) proxy * Added preview snippet for embedded content * Added support for optionally tagging links as 'nofollow' * Fixed missing css * Fixed move usual proxyIRI-handling above embedded fragments * Fixed issues with https in /describe content negotiation Alternates and Location headers * Fixed use https:// to work well with secure pages * Fixed issues loading insecure content into secure page * Added guard against shortning already short URLs * Added check to make sure page is in range 0 .. last * Fixed disable input fields to goto specific page if there is only a single page * Added function urilbl_ac_init_state to reflect state of Entity Data generation * Fixed typo * Fixed missing call to onchange so page refreshes when Show x rows selector changes * Fixed redirect URL encoding after sponging * Fixed URL encoding in pager * Fixed bad URL encoding in tabs * Disabled unwanted 'extra' decoding * Fixed use TTLP_V * Fixed bad check * Fixed i18n issue * Fixed do counts separately instead of using UNION to avoid issue with expected/generated columns error * Fixed error handling * Fixed issue "Value of ANY type column too long" with long shapes * Updated VAD version * Renamed binsrc/b3s to binsrc/fct * Revert "Fixed use TTLP_V" This reverts commit 33568d4. * Fixed missing variable declaration * Moved binsrc/yacutia binsrc/conductor * Fixed issue building with --debug-mtx * Updated version to 7.2.6-rc1 * Updated NEWS and ChangeLog in preparation for upcoming release * Fixed insufficient localization of errors on loading Turtle Closes: openlink#295 * Fixed GPFs on outermost-level empty shape (they're impossible on SHP and WKT/WKB reads but can be represented in EWKT) * Fixed version info * Fixed initialization when proj data is not available on the filesystem * Fixed version number * Fixed issue with PATCH command on collections * Added support for ',meta' for collections * Fixed issue with SSL HTTP authentication * Fixed missing cert argument in calls to DAV_AUTHENTICATE_SSL * Fixed check function names for invalid characters * Fixed merge error trying to call ACL_VALIDATE * Fix issues leaving too many open statement handles in server side JDBC/ODBC call * Updated JDBC driver version * Rebuild JDBC providers * Added missing header for ',meta' collections * Moved search controls in the main pane * Fixed issue linking against opensssl 1.0.1 or older * Fixed use function to set SSL accept state * Fixed missing EXPORT for proj4 plugin * Fixed check for LibreSSL version * Fixed portability issue with Alpine and other Linux versions that do not use glibc * Fixed portability issue using a C++ compiler * Update ODBC driver for support UTF16 * Fix SQLConnectW for UTF16 * Fix CLIw.c * Fixed stack overwrite * Fixed portability issue on Mac OS X * Fixed issues converting from SQL_C_CHAR to SQL_BINARY * Optimized OPTIONS method for WebDAV collections * Fixed output content requested by 'Accept: application/ld+json; profile='https://www.w3.org/ns/activitystreams' header * Fixed issue with creating LDP collection data * Fixed some issues based on the LDP testsuite execution http://w3c.github.io/ldp-testsuite/ * Fixed issue creating LDP collection data * Fixed issue with WebID selection * Updated Conductor VAD version * Fixed typo * Added support for property paths in federated SPARQL queries closes issue openlink#734 * Updated Copyright to include 2019 * Fix Jena3 provider for support Jena 3.10.0 * Rebuild Jena3 provider * Added separate links for text/n3 and text/turtle * Updated Memento timegate URL * Fixed issue with link concat order * Fix issues with Connection Option CHARSET=UTF-8 * Fixed issue with UTF-16 on SQLGetDiagRec * Fixed ODBC error string prefix * method VirtuosoConnection.isConnectionLost() was updated for use Statement instead of PreparedStatement * Updated JDBC driver to version 3.112 * Rebuild JDBC drivers * Updated version to 114 * Removed deprecated Virtuoso v5 DDL statements * Fixed typo * Added new bif function bif_boxint_range_arg * Fixed snprintf into buffers that are too short to always fit the output * Fixed issue with freeing fields * Fixed missing error check in sslt_qst_get * Fixed handling of DTD validation error in dtd_compile * Fixed misleading indentation * Added extra checks to blob test programs * Fixed issue getting .ttl file with curl * Fixed dav browser page size * Fixed issue with WebDAV HTTP Response Headers * Fixed issues with LDP implementation * Fixed memory leak due to bad allocation of GEO_POINTLIST shapes * Fixed missing args for MALLOC_DEBUG * Added support for JSON output * Fixed issue with checkpoint_interval(-1) * Added new function scheduler_interval() * Disable both checkpoint and scheduler when starting bulkload * Added missing headers for rdf-editor to browse DAV tree * Fixed keep existing options * Fixed use fully qualified names * Fixed use fully qualified names * Fixed login when conductor is behind a proxy * Fixed use fully qualified names * Fixed compiler warnings * Fixed do not try to run ttlp_v in atomic mode * Backported default_geo_type stub for r2rml * Fixed issue editing HTML and XSLT files with Chrome browser * Updated Conductor VAD version * Updated FCT VAD version * Fixed issue returning the correct client IP when using a proxy * Fixed remove box around uid and gid arguments * Fixed initialize struct * Fixed debug lines * Fixed return correct GEO type * Fixed issue with r2rml generating empty columns section * Fixed potential issue with cartesian product in R2RML * Fixed br to be wellformed * Fix issue with handle RDFFormat.BINARY in method RepositoryConnection.add() * Updated RDF4J version * Rebuild RDF4J provider * Fixed backslash escaped characters in QNames of SPARQL queries. (Bug16599) * Bug fixes - fixed count bug - fixed filter bug * Fixed missing variable declaration * Fixed missing grant * Fixed use new API calls for sponger * Fixed clear http session * Fixed use params for permalink * Fixed issues editing soap services * Update for ODBC driver for better integration with iODBC DM for support autoswitch between UCS4/UTF16 modes * Fix issue with insert XMLLiteral * Updated version of Jena 3 provider * Rebuild Jena 3 provider * DIsable Unix related block for fix compilation errors on Windows * Disabled txn check for backup dump Closes: openlink#578 * Fixed perform +restore-crash-dump in foreground * Fixed windows build issues for VS 2017 * Fixed typo * Fixed wrong percent range set * Fixed issue that enabled virtual paths content to be shown with PROPFIND command * Fixed issue with PATCH command and sparql updates * Fixed issue with ',acl' files * Fixed searching with tag conditions in DynaRes DETs * Added new optimizations for WebDAV * Added new column COL_FULL_PATH * Added support to move lost collections to '/DAV/.lost+found/' collection. * Added triggers to check the collection hierarchy before updates. * Added perfomance improvements for some often used functions. * Added additional checks for some API calls. * Updated triggers and procedures to use the new column COL_FULL_PATH * Fixed issue in conductor showing folder content after rename. * Fixed issue with ID of DET collections and optimize DAV_SEARCH_ID * Removed unused columns * Removed old v5 schema updates * Updated DAV_DIR_LIST API function to support limit and order parameters * Added new option PublicDebug to virtuoso.ini * New functions bif_st_n_points() and bif_st_point_n() * Disabled triggers generation for RDF view referencing SQL views * Conductor - updated default suggested ASK statement by IMAP filters. * Fixed issue with registry page - row exceeded max lenght * Fixed some warnings and simplified some pages by removing unneeded table tags * Fixed https based client * do not check server certificate when 'insecure' flag is set * register CA certificates when db:key is used * Fixed showing the path by virtual paths used by WebDAV browser * Fixed support for large limits for sizes of transactions * Fixed issue with timezone and scheduler view * Added icon support for some DET folder types * Added support for unfoldable internal functions in execution plan * Fixed syntax of '__tag of XXX' to support it in signed literals Added support for __tag of dictionary reference __tag of stream __tag of XXX handle * Replacing "magic numbers" of DV_xxx in SQL code with readable "__tag of". * Fixed issue with autocommit mode When row auto commit mode revert the non-txn insert flag back after vec insert. Otherwise sequence will be logged separately and txn rb image will grow a lot * Fixed issue with DML inside DDL statement * Fixed bug with dates without timezone by PROPFIND command * Fixed issue with xslt file used by WebDAV skin * Fixed issue with dates without timezone * Removed duplicate procedures which already exist in binary * Fixed issue with Accept header and *,meta files * Fixed missing files in build list * Updated list of extension to icon relations for WebDAV * Fixed issue mapping LDP enabled folders and PUT command * Fixed issue with HTTP commands in CalDAV and CardDAV protocols * Removed DAV_SEARCH_SOME_ID * Fixed issue deleting folder under version control * Added WebDAV max browser lines parameter * Fixed issue with bad filter in search function * Removed duplicate procedures used for validation * Fixed validation of boolean values * Fixed issue with popup WebDAV folder selection in filters * Fixed presentation and working with files under version control * Fixed issue with bad URL oin address bar by WebDAV browser * Added autofocus for the most used forms in WebDAV browser * Fixed labels * Added initial support for onBehalfOf * Added support for Escape key to close the WebDAV browser * Removed deprecated functions * Updated Conductor VAD version * Fixed issue with grouping: name could be dv_symbol * Added option to view CalDAV and CardDAV resources * Fixed getting listen interface for FTP server * Fixed issue in testsuite * Fixed issue in http testsuite 405 (Method is not allowed) - MKCOL can only be executed on an unmapped URL * Fixed issue with VXML The file:// must be used for local path (e.g. starting from www root) * Fixed issue with buffer size * Added parameter for ses params function * Updated FCT vad version * Fixed issue when login into ODS Closes: 652 * Updated ODS Framework VAD version * Fixed windows portability issue * Added support for insecure flag to SSL based soap call * Fixed compiler warnings * Fixed issue reading large blobs * Fixed issue with db_to_str_place * Fixed compiler warning * Added extra debug macros * Fixed compare canonical dtp for nvarchar for vec ssl casts * Fixed small issues in testsuite * Fixed issue building MALLOC_DEBUG * Fixed issue creating URIs by DAV meta subsystem. * Fixed output of PROPFIND command * FIxed issue returning correct value for WebDAV versioning * Fixed issue creating user's IRIs * Fixed issue checking auto version feature for resource * Added function to determine resource extension * Added qt_record error file and option to skip comments * Fixed issues in testsuite * Fixed make sure http headers are not send twice * Fixed portability issue * Updated Conductor VAD version * Added support for mkstemp * Fixed check for cli_session as bootstrap and scheduler do not have it * Fixed missing argument * Fixed dav option in crawler to get links * Added support for reporting current rss status * Fixed clear the association between target and carts list * Fixed syntax in export of crawler target * Updated Conductor version * Fix issues with POINTZ * Updated version of JDBC driver * Rebuild JDBC drivers * Fixed comment * Fixed issue with POST for application/sparql-query (w3)
|
I've just run into this again. Is there a plan for finally fixing this bug in Virtuoso? |
|
@pkleef, @IvanMikhailov, @kidehen, @openlink, @HughWilliams -- Any update on this issue? |
|
Hi, I just run into this issue and wonder why it is still not fixed? |
Hi. I'm running two Virtuoso instances, version
7.00.3203and version7.10.3211on a linux machine.I use virtuoso jdbc 3 to load geographical data like this:
7.00.3203, data is loaded properly without problem.7.10.3211, I get the following error:If I remove the property
asWKT, leaving just thelatandlongproperties and try to load it in version7.10.3211, data is loaded properly.I have checked data syntax and it is correct according to geosparql so I don't know what the problem is.
This issue is related with issue 274.
The text was updated successfully, but these errors were encountered: