Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeouts not working for exports #521

Open
rafatower opened this issue Jul 18, 2018 · 2 comments
Open

Timeouts not working for exports #521

rafatower opened this issue Jul 18, 2018 · 2 comments
Assignees

Comments

@rafatower
Copy link
Contributor

At lest the tests do not pass when I run against pgbouncer.

Steps to reproduce

  1. Run pgbouncer listening on port 6432.
  2. Modify the config/environments/test.js to set module.exports.db_port = '6432';
  3. Apply the patch in Allow tests to be executed against pgbouncer #520 because otherwise the tests die in the setup phase (explained there).
  4. Execute the tests like this:
$ make test 2>&1 | tee /tmp/test.out

cause at some point the diffs display the bodies containing SHP and other binary formats, which screw my terminal output, so that way I could inspect the /tmp/test.out file.

Expected result

All the tests pass.

Actual result

Here's an excerpt of the results I got:

  415 passing (38s)
  13 pending
  9 failing

  1) timeout export database Geopackage export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: SQLite format [...]

  2) timeout export database KML export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: <?xml version="1.0" encoding="utf-8" ?> [...]

  3) timeout export database Shapefile export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: [...]

(all the failures are related to exports)

@rafatower
Copy link
Contributor Author

(assigning to @dgaubert , for you to prioritize and dispatch at your leisure)

@dgaubert
Copy link
Contributor

Hey!

That's because PGBoucer has its internal pool of connection against Postgres, before executing those tests we need to set a proper timeout for that user/database. it uses one connection of PGBouncer pool but rest of them keep the connection with the previous settings and the timeout will not be applied.

In fact, we had to clean up connections from the internal cartodb-psql pool.

rafatower pushed a commit that referenced this issue Jul 19, 2018
And using sh can make some things fail. E.g:

```
[[: not found
```
rafatower pushed a commit that referenced this issue Jul 19, 2018
This sends a PAUSE and RESUME to pgbouncer (in case there's one) before
and after executing tests, to make sure new connections are established
in the tests.

This may be especially important when role or session settings are
modified in the DB (same happens in prod, BTW).
rafatower pushed a commit that referenced this issue Jul 19, 2018
This is needed to make the setup a little bit more robust: when trying
to delete the test database, it won't be able if there are "idle
sessions" in the db (that is, connections from pgbouncer to the test
database).

Otherwise it fails when trying to create the database, because there's
already one with the same name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants