Improve perf for large pending sets

[djmitche]

In #3329 @ashprice identified performance issues with a task DB containing 1800 pending tasks (and 4500 tasks total). Queries like task pri:5.000000 mod pri:5 take a looong time and ues a lot of RAM.

[felixschurk]

We can probably as a reference also here come back to the performance scripts in performance.

I just made a small run on my computer with only 100 tasks and this is the output of the v2.6.2 tag vs the current develop:

python3 compare_runs.py task2  task3 
Previous: 2.6.2 (7400e6ed6)
Current:  3.0.0 (8a0a98d3e)
# next:
 commit filter     gc hooks init  load other render sort total
    205   1505     76     0 3502  3081  4778   1699  263 15109
      0    825  23360     0 2720     0 17203   1321  310 45739
   -205   -680  23284     0 -782 -3081 12425   -378   47 30630
  -100%   -45% 30636%    0% -22% -100%  260%   -22%  17%  202%
# list:
 commit filter     gc hooks init  load other render sort total
    140   1097     71     0 5785  3207  2550   3258 1195 17303
      0    790  23611     0 5564     0 18191   2563  798 51517
   -140   -307  23540     0 -221 -3207 15641   -695 -397 34214
  -100%   -27% 33154%    0%  -3% -100%  613%   -21% -33%  197%
# all:
 commit filter     gc hooks init  load other render sort total
    131     89     61     0 3038  3579  1159   9688  389 18134
      0     50  19531     0 3119     0 19698  12116 1529 56043
   -131    -39  19470     0   81 -3579 18539   2428 1140 37909
  -100%   -43% 31918%    0%   2% -100% 1599%    25% 293%  209%
# add:
 commit filter gc hooks  init  load  other render sort  total
  16089      0  0     0  5661   802   1945      0    0  23695
      0      0  0     0  3524     0 156644      0    0 160168
 -16089      0  0     0 -2137  -802 154699      0    0 136473
  -100%     0% 0%    0%  -37% -100%  7953%     0%   0%   575%
# export:
 commit filter     gc hooks  init  load other render sort total
    145    190    149     0  3159  4303   921   4420    0 13287
      0     47  17435     0  1770     0 13897   4959  242 38350
   -145   -143  17286     0 -1389 -4303 12976    539  242 25063
  -100%   -75% 11601%    0%  -43% -100% 1408%    12%   0%  188%
# import:
 commit filter gc hooks  init  load other render sort total
  19170      0  0     0  2865  5369 22746      0    0 44781
      0      0  0     0  1862     0 81161      0    0 83023
 -19170      0  0     0 -1003 -5369 58415      0    0 38242
  -100%     0% 0%    0%  -35% -100%  256%     0%   0%   85%

To be honest I do not understand all the numbers which are displayed, however the percentage is quite clear.