Create re-design POC for catalog and datasets

[merelcht]

Introduction

Create a POC of a different way of having the functionality that's currently in the catalog and datasets.

Background

We discussed in technical design that both the catalog and datasets are fairly old components of Kedro. Knowing what we know about these parts, how could we make a better implementation?

What's in scope

• Create a re-design of the catalog and datasets
• Ideally this would be a POC

What's not in scope

• A fully fledged solution that will be merged into main. This is really meant as a design experiment.

The below sections can be used as a guideline of what to think about:

Design

Explain your design to the solution here. Diagrams could help.

Alternatives considered

Explain the trade off between different alternatives to your solution.

Testing

Explain the testing strategies to verify your design correctness (if possible).

Rollout strategy

Is the change backward compatible? If not, what is the migration strategy?

Future iterations

Will there be future iterations of this design?

[astrojuanlu]

What's the difference between this issue and #1778?

[merelcht]

@astrojuanlu I think #1778 is meant for gathering overall issues and discussion and this ticket for a concrete POC implementation.

[merelcht]

On further consideration I feel like this issue was created a bit pre-maturely and we need to first dig into #1778 more before we decide if a POC is the right way forward. So I'm closing this as not planned.