Horizon reaper uses the same db connection pool which is used to serve requests

[tamirms]

When the horizon reaper is initialized, it uses the same db connection pool which is used to serve Horizon requests:

go/services/horizon/internal/app.go

Lines 515 to 519 in f30d114

	a.reaper = reap.New(
	a.config.HistoryRetentionCount,
	a.config.HistoryRetentionReapCount,
	a.HorizonSession(),
	a.ledgerState)

This is problematic for two reasons:

1. SDF uses an architecture where ingesting horzon nodes are connected to a RW database and request serving horizon nodes are connected to a RO replica database. Reaping will fail in this scenario because delete statements will result in errors when executed on a read only database.
2. The db connection pool used for serving horizon requests is configured with a statement timeout which defaults to 10 seconds. The reaping DB statements are not guaranteed to execute within this time limit and in the scenario where there is a lengthy reap DB query it will block the progress of reaping.

To fix this issue we should only reap in Horizon when ingestion is enabled because that guarantees that we have a RW db connection. Also, we should create a new DB connection pool solely for the purpose of reaping. That connection pool should be configured to not have any statement timeouts. Or, if we decide to have statement timeouts for reaping, that configuration should be separate from the configuration for the statement timeout on the db queries connected to horizon HTTP requests.