savepoint & sqlalchemy errors

[awoods187]

I'm running Movr following the geo-partitioning example and hitting Savepoint retry errors after many hours of running the workload:

andrewwoods@andy-geo-partitioning-0001:~$ sudo docker run -it --rm cockroachdb/movr:19.03.2 --app-name "movr-east" --url "postgres://root@34.74.209.106:26257/movr?sslmode=disable" --num-threads=15 run --city="new york"
[INFO] (MainThread) connected to movr database @ postgres://root@34.74.209.106:26257/movr?sslmode=disable
[INFO] (MainThread) simulating movr load for cities ['new york']
[INFO] (MainThread) warming up....

Then I receive this error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: SAVEPOINT cockroach_restart needs to be the first statement in a transaction


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./loadmovr.py", line 388, in <module>
    run_load_generator(conn_string, args.read_percentage, args.city, args.echo_sql, args.num_threads)
  File "./loadmovr.py", line 327, in run_load_generator
    active_rides.extend(movr.get_active_rides(city))
  File "/usr/src/app/movr.py", line 115, in get_active_rides
    lambda session: get_active_rides_helper(session, city, limit))
  File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 31, in run_transaction
    return _txn_retry_loop(session, callback)
  File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 74, in _txn_retry_loop
    with _NestedTransaction(conn):
  File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 52, in __enter__
    self.conn.connection()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1120, in connection
    execution_options=execution_options,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1126, in _connection_for_bind
    engine, execution_options
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 432, in _connection_for_bind
    transaction = conn.begin_nested()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 656, in begin_nested
    self.__transaction = NestedTransaction(self, self.__transaction)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1754, in __init__
    self._savepoint = self.connection._savepoint_impl()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 774, in _savepoint_impl
    self.engine.dialect.do_savepoint(self, name)
  File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/dialect.py", line 425, in do_savepoint
    connection.execute('SAVEPOINT cockroach_restart')
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 974, in execute
    return self._execute_text(object_, multiparams, params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1147, in _execute_text
    parameters,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 276, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) SAVEPOINT cockroach_restart needs to be the first statement in a transaction
 [SQL: 'SAVEPOINT cockroach_restart'] (Background on this error at: http://sqlalche.me/e/f405)