Add Qdrant Provider

[Anush008]

This PR intends to add Qdrant as a supported provider by implementing an operator and hook that enables interfacing with Qdrant within Airflow DAGs.

Qdrant is an open-source, Apache-licensed, vector search engine and database.

[Discussion] Add Qdrant as a supported provider

[LAZY CONSENSUS] Adding an Airflow provider for Qdrant

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[hussein-awala]

I think that some of the hook tests are not needed since there is no wrapper for the client methods in the hook, you call them from the client and you check if they are called. Same for the hook integration tests. I don't have a strong opinion on this, let's wait for the other reviews.

[Anush008]

I think that some of the hook tests are not needed since there is no wrapper for the client methods in the hook, you call them from the client and you check if they are called. Same for the hook integration tests. I don't have a strong opinion on this, let's wait for the other reviews.

Right. There were some wrapper methods before this commit. But then they seemed redundant as the caller can very well get the client through get_conn() or @property conn and have access to all the methods. So I updated the mock hook tests accordingly.

[eladkal]

LGTM

[potiuk]

FYI: @Anush008 :

You will have to make your provider compatible with Airflow 2.7.0+ the compatibility checks detected, that your provider requires Pydantic 2 by, which is not supported in Airflow 2.6.*. You will need to specify apache-airflow>=2.6.0 in provider.yaml and exclude your provider for 2.6 checks in BASE_PROVIDER_COMPATIBILITY_CHECKS:

https://github.com/apache/airflow/blob/main/dev/breeze/src/airflow_breeze/global_constants.py#L467

[Anush008]

Hey.

the compatibility checks detected, that your provider requires Pydantic 2 by, which is not supported in Airflow 2.6.*.

qdrant_client uses Pydantic v1. The failing check compat seems to arise from Pydantic V2 use by a different provider.

[potiuk]

It's problem with Airflow - 2.6 has problem with Pydantic 2 (that's why you need to limit qdran to >= 2.7.0 because Airflow 2.6.* only supports Pydantic 1). That's why you cannot install qdrant on it. As of 2.7 Airflow supports both Pydantic 1/2 and in 2.8 only Pydantic >=2

[potiuk]

It's problem with Airflow - 2.6 has problem with Pydantic 1(that's why you need to limit it. That's why you cannot install qdrant on it. As of 2.7 airflwo supports both an d in 2.8. only >2

Or so it looked like(I now am using a phone)

[potiuk]

FINALLY GREEN !!!.

[potiuk]

LGTM

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[potiuk]

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

[Anush008]

Thanks @potiuk, @eladkal, @josh-fell, @hussein-awala for supporting this 🙏🙏.

Do let me know if there's anything else needed.

[potiuk]

Usually the real work starts after it gets in the hands of the users :)