Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When i use Debezium and wal2json it cause a problem #183

Closed
chzhty001 opened this issue Oct 15, 2020 · 3 comments
Closed

When i use Debezium and wal2json it cause a problem #183

chzhty001 opened this issue Oct 15, 2020 · 3 comments
Labels

Comments

@chzhty001
Copy link

The error info:
Caused by: org.postgresql.util.PSQLException: ERROR: out of memeory
Detail: Cannot enlarge string bugger containing 1073741788 bytes by 217 more bytes.
Where: slot "debezium", output plugin "wal2json", in the change callback, associated LSN xxxxxx

@eulerto
Copy link
Owner

eulerto commented Oct 16, 2020

You have a big transaction. format version 1 allocates one string for the whole transaction. The maximum size for format version 1 is 1 GB (this is a PostgreSQL infrastructure limitation). You have 3 options:

  • use smaller transactions and keep format version 1;
  • use format version 1 and enable write-in-chunks;
  • user format version 2.

I recommend that you adopt format version 2. I will be the default in future versions.

@chzhty001
Copy link
Author

  • format version 2.

how can i use format version 2? is it a config item?

@eulerto
Copy link
Owner

eulerto commented Oct 31, 2020

Parameter format-version. There is an example in the README.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants