Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PB-82: prepare migration to k8s #597

Merged
merged 3 commits into from
Jan 15, 2024
Merged

Conversation

ltclm
Copy link
Contributor

@ltclm ltclm commented Jan 12, 2024

a few changes in the dockerfile and in the efs sync script are needed during the parallel operations of service-search-sphinx on infra-vhost and k8s.

on k8s we are forced to mount the efs indexes with the full path: so this will be the folder with the efs index files:

  • /var/local/geodata/service-sphinxsearch/${DBSTAGING}/index

this is due to a limitation of the aws efs csi driver which is not supporting subpaths as volumemount config. we would have to create new peristant volumes with terraform which is unreasonable for this special use-case.

@ltclm ltclm requested review from rebert and ltshb January 12, 2024 13:13
@github-actions github-actions bot added this to the 2024-01-31 milestone Jan 12, 2024
@ltclm ltclm changed the title PB-82: a few changes in the dockerfile and in the efs sync script are PB-82: prepare migration to k8s Jan 12, 2024
@ltclm ltclm requested a review from ltflb-bgdi January 12, 2024 14:08
needed during the parallel operations of service-search-sphinx on
infra-vhost and k8s.

on k8s we are forced to mount the efs indexes with the full path:
so this will be the folder with the efs index files:
* /var/local/geodata/service-sphinxsearch/${DBSTAGING}/index
this is due to a limitation of the aws efs csi driver which is not
supporting subpaths as volumemount config. we would have to create new
peristant volumes with terraform which is unreasonable for this special
use-case.
@ltshb
Copy link
Contributor

ltshb commented Jan 15, 2024

I don't have the full picture here, but I'm wondering if it would not be better to have a dedicated EFS Access Point for sphinx index sharing instead of reusing the /prod and /dev of our geodata EFS ? We could reuse the geodata EFS but use another access point to better separate the search index from the geodata. This would not add more cost and would be clearer in the configuration. If sphinx needs also the geodata for indexing it can still mount both the geodata /prod|/dev and the new access point /service-search-sphinx-index-dev|int|prod

Copy link
Contributor

@ltflb-bgdi ltflb-bgdi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, looks good.

@ltclm
Copy link
Contributor Author

ltclm commented Jan 15, 2024

I don't have the full picture here, but I'm wondering if it would not be better to have a dedicated EFS Access Point for sphinx index sharing instead of reusing the /prod and /dev of our geodata EFS ? We could reuse the geodata EFS but use another access point to better separate the search index from the geodata. This would not add more cost and would be clearer in the configuration. If sphinx needs also the geodata for indexing it can still mount both the geodata /prod|/dev and the new access point /service-search-sphinx-index-dev|int|prod

@ltshb
thats a good point, i have added it to the open questions here:
https://github.com/geoadmin/infra-kubernetes/pull/482

Copy link
Contributor

@rebert rebert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ltclm
Copy link
Contributor Author

ltclm commented Jan 15, 2024

i will merge this now so that we can observe sphinx on k8s on dev under real-life conditions.

@ltclm ltclm merged commit 5dee937 into develop-2024-01-31 Jan 15, 2024
5 checks passed
@ltclm ltclm deleted the feat_PB-82_search_k8s branch January 15, 2024 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants