Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PGXS "make installcheck" support #28

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

df7cb
Copy link

@df7cb df7cb commented Jan 10, 2022

This enables running PG's standard testsuite with a PG server configured
for decoderbufs.

Example on Debian:

$ sudo apt install postgresql-14-decoderbufs
$ pg_virtualenv -o shared_preload_libraries=decoderbufs -o wal_level=logical
Creating new PostgreSQL cluster 14/regress ...
$ make installcheck
/usr/lib/postgresql/14/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/lib/postgresql/14/bin'    --dbname=contrib_regression decoderbufs
(using postmaster on localhost, port 5433)
============== dropping database "contrib_regression" ==============
HINWEIS:  Datenbank »contrib_regression« existiert nicht, wird übersprungen
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries        ==============
test decoderbufs                  ... ok           30 ms

=====================
 All 1 tests passed.
=====================

$ exit
Dropping cluster 14/regress ...

Tested on apt.postgresql.org with PG 10+ on amd64/arm64/i386/ppc64el.

https://pgdgbuild.dus.dg-i.net/view/Testsuite/job/postgres-decoderbufs-autopkgtest/

This enables running PG's standard testsuite with a PG server configured
for decoderbufs.

Example on Debian:

```
$ sudo apt install postgresql-14-decoderbufs
$ pg_virtualenv -o shared_preload_libraries=decoderbufs -o wal_level=logical
Creating new PostgreSQL cluster 14/regress ...
$ make installcheck
/usr/lib/postgresql/14/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/lib/postgresql/14/bin'    --dbname=contrib_regression decoderbufs
(using postmaster on localhost, port 5433)
============== dropping database "contrib_regression" ==============
HINWEIS:  Datenbank »contrib_regression« existiert nicht, wird übersprungen
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries        ==============
test decoderbufs                  ... ok           30 ms

=====================
 All 1 tests passed.
=====================

$ exit
Dropping cluster 14/regress ...
```
@df7cb
Copy link
Author

df7cb commented Sep 24, 2024

Is anyone listening here? PG 17 will be released on Thursday, and decoderbufs isn't ready yet:

make[1]: Verzeichnis „/home/myon/projects/postgresql/postgres-decoderbufs/postgres-decoderbufs“ wird betreten
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -g -O2 -Werror=implicit-function-declaration -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fno-omit-frame-pointer -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/home/myon/projects/postgresql/postgres-decoderbufs/postgres-decoderbufs=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fPIC -fvisibility=hidden -std=c11  -I/usr/local/include  -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o src/decoderbufs.o src/decoderbufs.c
src/decoderbufs.c: In function ‘pg_decode_change’:
src/decoderbufs.c:625:53: error: ‘HeapTupleData’ has no member named ‘tuple’
  625 |                            &change->data.tp.newtuple->tuple, tupdesc);
      |                                                     ^~
src/decoderbufs.c:631:53: error: ‘HeapTupleData’ has no member named ‘tuple’
  631 |                            &change->data.tp.newtuple->tuple, tupdesc);
      |                                                     ^~
src/decoderbufs.c:646:55: error: ‘HeapTupleData’ has no member named ‘tuple’
  646 |                              &change->data.tp.oldtuple->tuple, tupdesc);
      |                                                       ^~
src/decoderbufs.c:656:55: error: ‘HeapTupleData’ has no member named ‘tuple’
  656 |                              &change->data.tp.newtuple->tuple, tupdesc);
      |                                                       ^~
src/decoderbufs.c:662:55: error: ‘HeapTupleData’ has no member named ‘tuple’
  662 |                              &change->data.tp.newtuple->tuple, tupdesc);
      |                                                       ^~
src/decoderbufs.c:678:53: error: ‘HeapTupleData’ has no member named ‘tuple’
  678 |                            &change->data.tp.oldtuple->tuple, tupdesc);
      |                                                     ^~
make[1]: *** [<eingebaut>: src/decoderbufs.o] Fehler 1

@gunnarmorling
Copy link
Member

Paging @jpechane :)

Most attention is on pgoutput these days, though, and it's the recommended decoding plug-in when available.

@df7cb
Copy link
Author

df7cb commented Sep 24, 2024

Is that the pgoutput.so shipped with contrib? That's available on every Debian install. If that is strictly better, we could also drop the Debian package of decoderbufs and go with pgoutput instead.

@jpechane
Copy link

@df7cb Hi, we definitely want to keep protobuf for some time. We'll make pgoutput default option sometimes during 3.x timeline and deprecate protbuf later depending on community feedback.

@jpechane
Copy link

@df7cb Please see https://issues.redhat.com/browse/DBZ-8275

@df7cb
Copy link
Author

df7cb commented Oct 27, 2024

3.0.1 doesn't compile here against PG17:

src/decoderbufs.c: In function ‘pg_decode_change’:
src/decoderbufs.c:631:28: error: passing argument 3 of ‘tuple_to_tuple_msg’ from incompatible pointer type [-Wincompatible-pointer-types]
  631 |                            &change->data.tp.newtuple TUPLE_ACCESS, tupdesc);
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~
      |                            |
      |                            HeapTupleData **
src/decoderbufs.c:396:61: note: expected ‘HeapTuple’ {aka ‘HeapTupleData *’} but argument is of type ‘HeapTupleData **’
  396 |                                Relation relation, HeapTuple tuple,
      |                                                   ~~~~~~~~~~^~~~~

Also, could you perhaps enable GitHub issues on this repository? It's the only one I know that has only pull requests enabled, but not issues.

@jpechane
Copy link

@df7cb We do use Jira exclusively. It would be a bit of complications for us to track issue accross mutliple issue trackers.

@bradfordboyle
Copy link

@df7cb PR #32 should address the compilation error with PG17. I've tested this change on trixie with both PG16 and PG17 using the installcheck test that you've added in this PR.

@jpechane
Copy link

@df7cb Hi, I am sorry I am necormancing this PR. I'd like to get it in but if possible also add a github workflow that would run on each PR and push to main. The workflow would build the plguin and run the installcheck on it. Would it be possible?

@df7cb
Copy link
Author

df7cb commented Nov 13, 2024

@df7cb PR #32 should address the compilation error with PG17. I've tested this change on trixie with both PG16 and PG17 using the installcheck test that you've added in this PR.

Thanks, the Debian packages are happy now.

@df7cb
Copy link
Author

df7cb commented Nov 13, 2024

@df7cb Hi, I am sorry I am necormancing this PR. I'd like to get it in but if possible also add a github workflow that would run on each PR and push to main. The workflow would build the plguin and run the installcheck on it. Would it be possible?

There's various extensions doing that, for example https://github.com/df7cb/postgresql-unit/blob/master/.github/workflows/regression.yml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants