Better dynamic runtime queue selection support

[prabirshrestha]

My app choose dynamically between sqlite and postgres. Is it possible to support AnyPool from sqlx instead of using only SqlitePool for SqliteQueue::with_pool(anypool)?

I would also like to store queue in struct but it currently requires specifying JobHandle?

pub struct AppState {
  queue: Arc<Queue>,
}

let queue = match anypool.any_kind() {
    AnyKind::Postgres => PostgresQueue::with_pool(anypool),
    AnyKind::Sqlite => SqliteQueue::with_pool(anypool),
    AnyKind::MySql => MySqlQueue::with_pool(anypool),
}

let appstate = AppState { queue };

For now it doesn't matter much since aide-de-camp only supports sqlite, but would like to make it work for generic cases where I can easily swap the queue.

[andoriyu]

@prabirshrestha AnyPool doesn't work with query! macros. That's the main obstacle. Also, for PostgreSQL, the plan is to use NOTIFY and some other PostgreSQL features. That only works with Pool<Postgres>.

I would also like to store queue in struct but it currently requires specifying JobHandle?

I haven't thought about this. In our codebase, Queue is stored in database-specific type, so this issue never surfaced. We might be able to make JobHandle into a concrete type and move functionality into Queue itself, therefore removing the associated type from Queue completely.

However, Queue, I think, trait isn't object safe anyway because all useful functions are generic over job type submitted.

[prabirshrestha]

It is a bummer that it wouldn't work. In the mean time I have opened launchbadge/sqlx#2162.

let queue: AnyQueue = match anypool.any_kind() {
   AnyKind::Postgres => PostgresQueue::with_pool(any_pool.try_into().unwrap()).into(),
   AnyKind::Sqlite => SqliteQueue::with_pool(any_pool.try_into().unwrap()).into();
}

Might be similar like sqlx, instead of having generic Queue, it should have AnyQueue? For those that want to support multiple they would use AnyQueue and they can always go back to PostgresQueue or SqliteQueue if they need more control.

[andoriyu]

Yeah, I think having a AnyQueue is a reasonable way to go. Ideally there will be a macro that can generate AnyQueue and AnyJobHandler style enums with queues you want to support instead of a single "official" one. In the end we want to support backends beyond what sqlx supports: Redis, NATS, Kafka, etc. it's just currently our internal need is just SQLite with PostgreSQL on the horizon.

…

On Wed, Oct 19, 2022, 8:56 PM Prabir Shrestha ***@***.***> wrote: It is a bummer that it wouldn't work. In the mean time I have opened launchbadge/sqlx#2162 <launchbadge/sqlx#2162>. let queue: AnyQueue = match anypool.any_kind() { AnyKind::Postgres => PostgresQueue::with_pool(any_pool.try_into().unwrap()).into(), AnyKind::Sqlite => SqliteQueue::with_pool(any_pool.try_into().unwrap()).into();} Might be similar like sqlx, instead of having generic Queue, it should have AnyQueue? For those that want to support multiple they would use AnyQueue and they can always go back to PostgresQueue or SqliteQueue if they need more control. — Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEFG6PYB734IH3WORZODLWEC7G7ANCNFSM6AAAAAARIVKELY> . You are receiving this because you commented.Message ID: ***@***.***>