[DataCatalog2.0]: Extend KedroDataCatalog with dict interface

[ElenaKhaustova]

Description

As a continuation of a work on DataCatalog redesign (#3995 (comment)) we suggest simplifying the interface for KedroDataCatalog and make it similar to UserDict

The reasoning: this way we want to make the catalog functioning as a collection of datasets with an interface familiar to users.

Related tickets that will be addressed:

• [DataCatalog]: Iterate through datasets objects in the catalog #3916
• [DataCatalog]: Revise methods and properties names for DataCatalog #3931

Context

Possible Implementation

Change the KedroDataCatalog API to support __iter__, __getitem__, __setitem__, keys(), items() similar to the dict.

Replace old API such as save(), load(), list(), etc.

The first iteration can be done without breaking changes if keep old interface and reuse newly added methods.

[datajoely]

I'm actually less excited about this than a lot of the other proposed ideas for this redesign.

What about putting it inside a property or something like a .dict_like() method?

[ElenaKhaustova]

I'm actually less excited about this than a lot of the other proposed ideas for this redesign.

What about putting it inside a property or something like a .dict_like() method?

That's an attempt to align on the interface changes before introducing new features, so it doesn't cancel the rest of the redesign idea. But since this is a proposal of a possible interface, it will be helpful at this stage if you can expand your thoughts about why it feels not good.

[noklam]

Related idea discussed in the past: #3914 (comment)

[astrojuanlu]

We discussed this on backlog grooming. To clarify, the user impact of this one is that rather than doing

>>> catalog.list()
>>> catalog.load("my_dataset")
>>> catalog.save("my_dataset", data)
>>> for dataset in catalog.datasets: pass

the user would do

>>> list(catalog)  # __iter__
>>> catalog["my_dataset"]  # __getitem__
>>> catalog["my_dataset"] = data  # __setitem__
>>> for dataset in catalog: pass  # __iter__

While I'm not opposed to have the dict-like interface, I think it's worth discussing whether we should keep the method-based interface. We're not forced to keep it, because it's a new class and we allow ourselves to do breaking changes between minor releases. But if we do include it, I expect migration between 0.19.* and 0.20.* will definitely be easier.