Skip to content

Commit

Permalink
feat(db): batch cleanup expired rows from postgres
Browse files Browse the repository at this point in the history
### Summary

The PR #10405 changed cleanup to only happen on nodes that have Admin API listeners.

This PR makes deletion to happen in maximum of 50.000 row batches.

Inspired from #10331 and #10389.
  • Loading branch information
bungle committed Mar 1, 2023
1 parent 5527939 commit 1740809
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 7 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
#### Core

- Postgres TTL cleanup timer will now only run on traditional and control plane nodes that have enabled the Admin API.
- Postgres TTL cleanup timer now deletes maximum of 50.000 rows per table per cleanup round.

## 3.2.0

Expand Down
18 changes: 11 additions & 7 deletions kong/db/strategies/postgres/connector.lua
Original file line number Diff line number Diff line change
Expand Up @@ -325,13 +325,17 @@ function _mt:init_worker(strategies)
local table_name = table_names[i]
local column_name = table_name == "cluster_events" and expire_at_escaped
or ttl_escaped
cleanup_statements[i] = concat {
" DELETE FROM ",
self:escape_identifier(table_name),
" WHERE ",
column_name,
" < CURRENT_TIMESTAMP AT TIME ZONE 'UTC';"
}
local table_name_escaped = self:escape_identifier(table_name)

cleanup_statements[i] = fmt([[
WITH rows AS (
SELECT ctid
FROM %s
WHERE %s < CURRENT_TIMESTAMP AT TIME ZONE 'UTC'
ORDER BY %s LIMIT 50000 FOR UPDATE SKIP LOCKED)
DELETE
FROM %s
WHERE ctid IN (TABLE rows);]], table_name_escaped, column_name, column_name, table_name_escaped)
end

local cleanup_statement = concat(cleanup_statements, "\n")
Expand Down

0 comments on commit 1740809

Please sign in to comment.