Cache using Ceph

[flowrean]

Hi

I am trying to use Ceph for caching using the S3 cache provider. Here's what my config looks like:

[cache]
type = "s3"
endpoint = "my-endpoint-ceph"
bucket = "my-bucket"
basepath = "my-basepath"
aws_access_key_id = "${CONFIG_S3_ACCESSKEY}"
aws_secret_access_key = "${CONFIG_S3_SECRETKEY}"

Tegola fails with the following message:

could not register cache: cache: error setting to (s3) cache: RequestError: send request failed
caused by: Put "https://my-bucket.my-endpoint-ceph/my-basepath/tegola-test-map/test-layer/0/0/0": dial tcp: lookup my-bucket.my-endpoint-ceph: no such host

Isn't Ceph supported or am I doing something wrong? I'm not sure if the URL looks correct.

[ARolek]

@flowrean does Ceph use the same API as S3? I know minio does, but I haven't played with Ceph directly.

[flowrean]

@ARolek yes, I have also used the aws-sdk-go to upload files.

[ARolek]

@flowrean to the domain in your error log? If the API is S3 compliant then it should work fine. It seems like tegola can't reach the host.

[flowrean]

@ARolek Yeah, I used the same variables (except another basepath). But I think the URL that is formed is different. Looking at the Ceph docs I would expect my-endpoint-ceph/my-bucket would have to be used instead of my-bucket.my-endpoint-ceph. Does that make sense?

[flowrean]

Ah, I think I used DisableEndpointHostPrefix previously. It probably isn't possible to set this via the Tegola config?

[ARolek]

@flowrean have you looked at the available config params for the s3 cache? I know it has been enhanced previously to support Minio. If you have already reviewed the configs (I didn't see anything obvious that would support DisableEndpointHostPrefix) and don't see anything that would work, we might need to enhance the cache provider. Would you be interested in investigating this?

[flowrean]

I have looked at the S3 config parameters indeed. Will see if I can get it working with DisableEndpointHostPrefix and possibly submit a PR.

[flowrean]

It turned out I could use the force_path_style config property (DisableEndpointHostPrefix apparently wasn't necessary). So no code changes were required. But I had to look in the code to find that property; it wasn't mentioned in the README. Hence my small PR to update the S3 config docs.

[flowrean]

Also, your website only lists an even smaller subset of these parameters: https://tegola.io/documentation/configuration/#s3

[ARolek]

@flowrean yeah the tegola docs need some love. It would be helpful if you could send in a PR to update the s3 cache details there too. Eventually I would like to parse the readmes from the various packages and turn them into the docs.