A large number of Spring Batch Jobs would lead to the exhaustion of database connections?!

[wondywang]

Recently, I've been considering developing a batch processing system using Spring Batch. However, while looking into the Spring Batch architecture, I found a potential issue.

Each batch job depends on database records for task information. Through use cases, it was verified that one Job establishes 11 long connections to the database. So, if I'm designing a Spring Batch system where all Jobs connect to the same database, does this limit the number of Jobs that can run in parallel?

For example, if 1000 Jobs are running at the same time, would these 1000 Jobs establish 11,000 long connections to the database? This could quickly exhaust the database connection pool, leading to systemic failures.

I hope my understanding is incorrect.

[Sax388]

What do you need the parallel processing for? Did you read https://docs.spring.io/spring-batch/reference/scalability.html? I don't think that this is a spring batch specific issue.

Note also that there may be limits placed on concurrency by any pooled resources used in your step, such as a DataSource. Be sure to make the pool in those resources at least as large as the desired number of concurrent threads in the step.

What do you think?