Implement Registry Multi-tenancy with Cognito in the loop

[jordanpadams]

💡 Description

Initial design and Serverless, schema setup:

• Design authorization handshake with Cognito, API Gateway, and Lambda for multi-tenancy approach #179
• Initial OpenSearch Serverless Setup and Deployment in MCP #223
• Search for a way to have OpenSearch aliases so that registry-dd can be accessed through <node>-registry-dd name #268

OpenSearch Serverless Authentication

• Implement Cloud Resources for OpenSearch Serverless Authentication #258

Tools adaptations

Loader Tools

• Update to utilize new multi-tenancy approach registry-mgr#66
• Update to utilize new multi-tenancy approach harvest#118
• Add schema reference to all examples to take advantage of new schema harvest#156

API

• Update to utilize new multi-tenancy approach registry-api#304
• As a web developer, I want to access read-only any feature provided by opensearch registry-api#433

Sweeper

• Registry-sweeper upgrade for multitenant registry registry-sweepers#120
• Registry-Sweeper ECS Enchancements (Post Multi-Tenancy) registry-sweepers#60
• Implement ECS task concurrency prevention for registry-sweepers planetary-data-cloud#105

Other

• As a data manager, I want to query OpenSearch Serverless from the command-line #273

Additional Testing

• Respond to User Questions Regarding Supporting Migration to OpenSearch Serverless #275
• Upgrade validate-refs to work with new AOSS registry validate#621
• TBD Deep Archive smoke test
• Upgrade docker compose as needed for the multitenant setup #269
• Smoke tests to ensure the migration is successful #279

Deployment and Data Migration

• Deploy Multi-tenant Registry to production and sync data #271
• Deploy Multi-tenant Registry to test #272
• Create snapshots of production OpenSearch clusters prior to shutdown #278
• Copy data necessary for web mod efforts to search-en registry prior to shutdown #277
• Update registry documentation #288

Cutover planning

• Write a migration plan to multitenant registry #217

[jordanpadams]

📆 April Status: Completion date expected to be delayed 1 week due to technology comparison needed. No impact on other tasks or deliveries

[jordanpadams]

📆 May status: Delayed 1 sprint due to increase in scope and complexity. Solution has been identified and is underway. Slack built into schedule should alleviate delays. No impact on delivery expected.

[jordanpadams]

📆 June status: Delayed an additional sprint due to increasing scope and learning-on-the-fly with OpenSearch authorization. No impact on build anticipated.

[jordanpadams]

📆 July status: Long delay due to Registry / API issues causing block in #179. Full deployment with authorization layer to be delayed, but partial deployment expected by delivery date. This delivery will actually cause less impact on users, so the partial deployment will have no effect on success of delivery.

[jordanpadams]

📆 August status: Working this architecture and deployment with MCP SAs. Expected completion by end of month.

[jordanpadams]

📆 September status: tightly coupled with NASA-PDS/planetary-data-cloud#35, this is in work. Continuing to work issues with serverless approach and AWS team.

[jordanpadams]

📆 October Status: tightly coupled with NASA-PDS/planetary-data-cloud#35, this is in work, but blocked by AWS technical team response times and issues being encountered with the service

[jordanpadams]

📆 04/2024 status: Delayed an additional 2 sprints. Will require separate mini-I&T to complete this delivery and release.

[jordanpadams]

📆 05/2024 status: Delayed 1 additional sprint. Will require separate mini-I&T to complete this delivery and release. Outside of scope of B14.1 delivery.

[jordanpadams]

📆 06/2024 status: Delayed an additional sprint due to additional unforeseen blocking issues. Significant impacts on rest of tasks in Build. All impacts will be review with SWG.

[jordanpadams]

📆 07/2024 status: Progress ongoing, but delayed an additional 2 sprints due to additional unforeseen blocking issues with networking. Significant impacts on rest of tasks in Build. All impacts will be reviewed with SWG.

[jordanpadams]

📆 08/2024 status: Progress ongoing, but delayed an additional sprint. To be done by end of this sprint

[jordanpadams]

This is done. Moving remaining sub-tasks to next build since they are some wrap-up activities that are not required for delivery to production.