[YSQL] yb_read_from_followers not used when set in function

[FranckPachot]

Jira Link: DB-9486

Description

It is possible to use follower reads for a specific query by using a function that sets the required parameters. However, it seems that yb_read_from_followers is not utilized.

Example:

-- install YBWR to get per-tablet metrics
\! curl -s https://raw.githubusercontent.com/FranckPachot/ybdemo/main/docker/yb-lab/client/ybwr.sql | grep -v '\watch' > ybwr.sql
\i ybwr.sql

-- create a table (on a RF3 cluster to have followers)
create table demo as select * from generate_series(1,6);

-- create a function to read the table and show the parameters
create or replace function f() returns table (transaction_read_only text, yb_read_from_followers text) as $$
 select current_setting('transaction_read_only'), current_setting('yb_read_from_followers') from demo;
$$ language sql;
alter function f set transaction_read_only=on;
alter function f set yb_read_from_followers=on;

-- call the function and show where it is read from
execute snap_reset;
select * from f() ;
execute snap_table;

-- set follower reads for the whole session and run the same
set yb_read_from_followers=on;
execute snap_reset;
select * from f() ;
execute snap_table;

The first run shows that yb_read_from_followers is on but reads only from the leaders:

yugabyte=# execute snap_reset;
 ybwr metrics
--------------
(0 rows)

yugabyte=# select * from f() ;
 transaction_read_only | yb_read_from_followers
-----------------------+------------------------
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
(6 rows)

yugabyte=# execute snap_table;
 I-seek | I-next | I-prev | R-seek | R-next | R-prev | insert | dbname relname tablet (L)eader node | (I)ntents (R)regular
--------+--------+--------+--------+--------+--------+--------+------------------------------------------------------------
        |        |        |      1 |        |        |        | yugabyte demo hash_split: [0x0000, 0x5554] L 10.0.0.40
        |        |        |      1 |      2 |        |        | yugabyte demo hash_split: [0x5555, 0xAAA9] L 10.0.0.39
        |        |        |      1 |      4 |        |        | yugabyte demo hash_split: [0xAAAA, 0xFFFF] L 10.0.0.41
(3 rows)

The second run set yb_read_from_followers at session level and reads from followers in the same node (10.0.0.39):

yugabyte=# set yb_read_from_followers=on;
SET
yugabyte=# execute snap_table;
 I-seek | I-next | I-prev | R-seek | R-next | R-prev | insert | dbname relname tablet (L)eader node | (I)ntents (R)regular
--------+--------+--------+--------+--------+--------+--------+------------------------------------------------------------
(0 rows)

yugabyte=# select * from f() ;
 transaction_read_only | yb_read_from_followers
-----------------------+------------------------
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
(6 rows)

yugabyte=# execute snap_table;
 I-seek | I-next | I-prev | R-seek | R-next | R-prev | insert | dbname relname tablet (L)eader node | (I)ntents (R)regular
--------+--------+--------+--------+--------+--------+--------+------------------------------------------------------------
        |        |        |      1 |        |        |        | yugabyte demo hash_split: [0x0000, 0x5554]   10.0.0.39
        |        |        |      1 |      2 |        |        | yugabyte demo hash_split: [0x5555, 0xAAA9] L 10.0.0.39
        |        |        |      1 |      4 |        |        | yugabyte demo hash_split: [0xAAAA, 0xFFFF]   10.0.0.39
(3 rows)

A workaround is to set yb_read_from_followers before the transaction starts but then all read only transactions will do stale reads, which is not desirable.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.

[FranckPachot]

Adding that the same behavior is observed when yb_read_from_followers is set within the transaction:

yugabyte=# execute snap_reset;
 ybwr metrics
--------------
(0 rows)

yugabyte=# set yb_read_from_followers=off;
SET
yugabyte=# begin transaction;
BEGIN
yugabyte=*#  set local yb_read_from_followers=on;
SET
yugabyte=*#  select * from f() ;
 transaction_read_only | yb_read_from_followers
-----------------------+------------------------
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
 on                    | on
(6 rows)

yugabyte=*#  commit;
COMMIT
yugabyte=# execute snap_table;
 I-seek | I-next | I-prev | R-seek | R-next | R-prev | insert | dbname relname tablet (L)eader node | (I)ntents (R)regular
--------+--------+--------+--------+--------+--------+--------+------------------------------------------------------------
        |        |        |      1 |      3 |        |        | yugabyte demo hash_split: [0x0000, 0x5554] L 10.0.0.40
        |        |        |      1 |      3 |        |        | yugabyte demo hash_split: [0x5555, 0xAAA9] L 10.0.0.41
        |        |        |      1 |        |        |        | yugabyte demo hash_split: [0xAAAA, 0xFFFF] L 10.0.0.39
(3 rows)

[jameshartig]

I believe the fix for this caused a regression filed here: #25635.