docs: add Daft integration

[avriiil]

This adds an integration page for using Delta Lake with Daft.

[ion-elgreco]

Not so sure about promoting daft here, they collect telemetry by default...

[MrPowers]

Seems like their telemetry gathering is responsible, but open to thoughts: https://www.getdaft.io/projects/docs/en/latest/faq/telemetry.html

[ion-elgreco]

I don't really agree, no one is going to know telemetry is on when you pip install it.

They likely also violate laws and regulations for users falling under GDPR regulation

[MrPowers]

@ion-elgreco - I don't think that non-identifiable data falls under the scope of GDPR.

[jaychia]

Hey folks! I'm one of the maintainers of the Daft project, just chiming in here to help clarify some of the work in Daft.

All telemetry collected is non-identifiable, which should not fall in scope of GDPR.

We also noticed that other frameworks which delta-rs already integrates with and currently includes in its documentation also do collect telemetry (though maybe they are not as public as we are so it is less discoverable).

• Ray: https://docs.ray.io/en/latest/cluster/usage-stats.html
• Dagster: https://docs.dagster.io/about/telemetry

I think that this is important to call out because I wanted to keep discussions on this PR related to the integrations with Daft, and perhaps if the Delta project needs to discuss its stance on its integrations' telemetry policies that can be discussed in a separate thread/issue?

Happy to provide any more clarifications if necessary!

[avriiil]

Hey @ion-elgreco @MrPowers - curious on how we can best proceed here? Would be great to get the integration docs in so that folks who read this new Delta blog can access the relevant documentation :)

[ion-elgreco]

I would rather see a mention on all these integrations that they collect telemetry which can be opted out from, just a small note at the bottom..

[avriiil]

@ion-elgreco - that seems fine to me. Perhaps out of scope for this PR since it will affect the docs for the other integration techs too?

[avriiil]

@ion-elgreco I've gone ahead and added a note at the end. LMK what you think :)

[avriiil]

Thanks for your help getting this through @ion-elgreco!