Preserve relative urls in catalog references

[ScottWales]

The leading '/' in catalog references is needed to construct the full URL path from a catalog reference.

For instance, in the catalog http://thredds-test.unidata.ucar.edu/thredds/testDatasets.xml the reference TestFmrc points to /thredds/catalog/fmrc/ecmwf/global_2p5/catalog.xml. Resolving the relative URL should give http://thredds-test.unidata.ucar.edu/thredds/catalog/fmrc/ecmwf/global_2p5/catalog.xml. Removing the /, as is being done in CatalogRef.__init__(), means the URL is constructed relative to the current catalog path, rather than being relative to the hostname.

[dopplershift]

In principle I have no problem with this, but I see @lesserwhirls is responsible for the lines in question, so I'd like a second from him.

Thanks for the contribution and tests!

[lesserwhirls]

First off, I echo @dopplershift - thank you @ScottWales for the contribution and tests!

It's a minor change, and I don't see anything wrong there. At some point soon, however, we should probably look at the module urlparse (python2, in 3 it is urllib.parse) to handle joining the fragments of the url.

We can accept this as is as far as I am concerned just to get it working now. That said - @ScottWales, are you feeling frisky, because you could try to use urlparse in the fix to see how it goes.

Python 2:

import urlparse
urlparse.urljoin(base, catalog_ref)

Python 3:

import urllib.parse
urllib.parse.urljoin(base, catalog_ref)

Note that catalog_ref can be a relative path, or a fully qualified path. If relative, it is joined to base; if fully qualified, it is used as is. So a bonus of using urlparse is that it will "do the right thing" under the hood if the for relative and fully qualified catalogRefs.

[ScottWales]

Happy to help. I've modified the pull request so that CatalogRef.href now stores the absolute URL to the target catalog, calculated using urljoin from the catalog's base URL.

I've also added the CatalogRef.follow() function mentioned in #20 since it has a trivial implementation, and updated the test case to use it.

[landscape-bot]

Code quality remained the same when pulling 1f053a3 on ScottWales:relative-url into c846074 on Unidata:master.

[lesserwhirls]

That's great @ScottWales - thank you!