Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] MCTSForest #2307

Merged
merged 12 commits into from
Nov 8, 2024
Merged

[Feature] MCTSForest #2307

merged 12 commits into from
Nov 8, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 22, 2024 •
edited
Loading

@vmoens
Update
3a6ada7 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Jul 22, 2024
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 22, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2307

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 13 Unrelated Failures

As of commit 570afff with merge base 0eabb78 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Jul 22, 2024
Merged
Merged
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 22, 2024
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] MCTSForest
dda9371 
ghstack-source-id: efee967055c8dab87bbef442451ce20f21730b13
Pull Request resolved: #2307
@vmoens vmoens added the enhancement New feature or request label Jul 22, 2024
@vmoens
Update
5218277 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 23, 2024
@vmoens
[Feature] MCTSForest
d1bc98a 
ghstack-source-id: 761863077202685046b093070361970c88081e36
Pull Request resolved: #2307
@vmoens
Update
f985808 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 3, 2024
@vmoens
[Feature] MCTSForest
a644a44 
ghstack-source-id: 0ea1c317017400ec27cc4620766f5c1ec413034d
Pull Request resolved: #2307
@vmoens
Update
a382e65 
[ghstack-poisoned]
This was referenced Aug 4, 2024
Merged
Open
Open
Closed
vmoens added a commit that referenced this pull request Aug 7, 2024
@vmoens
[Feature] MCTSForest
6deedec 
ghstack-source-id: dd7d9393589e9e13de6e7f61a019992b4efe8ee2
Pull Request resolved: #2307
@vmoens
Update
9d95a2b 
[ghstack-poisoned]
@vmoens
Update
1d8f21f 
[ghstack-poisoned]
@vmoens
Update
e3742c8 
[ghstack-poisoned]
@vmoens
Update
50d0c3a 
[ghstack-poisoned]
@vmoens
Update
46d798f 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 4, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}11$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4286s 0.4247s 2.3548 Ops/s 2.2828 Ops/s $\color{#35bf28}+3.16\%$
test_transformed 0.6778s 0.6069s 1.6478 Ops/s 1.7286 Ops/s $\color{#d91a1a}-4.68\%$
test_serial 1.3425s 1.3290s 0.7525 Ops/s 0.7453 Ops/s $\color{#35bf28}+0.96\%$
test_parallel 1.2853s 1.2734s 0.7853 Ops/s 0.7784 Ops/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-True-True-True-True] 0.1386ms 26.4287μs 37.8377 KOps/s 36.5441 KOps/s $\color{#35bf28}+3.54\%$
test_step_mdp_speed[True-True-True-True-False] 44.0830μs 15.3465μs 65.1613 KOps/s 62.2122 KOps/s $\color{#35bf28}+4.74\%$
test_step_mdp_speed[True-True-True-False-True] 56.5760μs 14.9999μs 66.6673 KOps/s 64.3221 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-True-True-False-False] 33.0920μs 8.7751μs 113.9583 KOps/s 112.2639 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-False-True-True] 65.1810μs 28.4880μs 35.1025 KOps/s 34.1705 KOps/s $\color{#35bf28}+2.73\%$
test_step_mdp_speed[True-True-False-True-False] 44.7840μs 16.9865μs 58.8704 KOps/s 56.5063 KOps/s $\color{#35bf28}+4.18\%$
test_step_mdp_speed[True-True-False-False-True] 62.2560μs 16.6616μs 60.0184 KOps/s 57.9050 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-True-False-False-False] 34.3640μs 10.4271μs 95.9041 KOps/s 93.7539 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[True-False-True-True-True] 71.9640μs 30.0351μs 33.2944 KOps/s 32.1036 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[True-False-True-True-False] 61.8860μs 18.7780μs 53.2538 KOps/s 51.1167 KOps/s $\color{#35bf28}+4.18\%$
test_step_mdp_speed[True-False-True-False-True] 41.6480μs 16.8716μs 59.2711 KOps/s 58.4490 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-False-True-False-False] 35.4860μs 10.3637μs 96.4905 KOps/s 94.0987 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-False-True-True] 72.8250μs 31.7143μs 31.5315 KOps/s 30.8283 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-False-False-True-False] 63.7100μs 20.1198μs 49.7022 KOps/s 47.3859 KOps/s $\color{#35bf28}+4.89\%$
test_step_mdp_speed[True-False-False-False-True] 70.0140μs 18.2015μs 54.9405 KOps/s 52.9322 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[True-False-False-False-False] 33.1320μs 11.9854μs 83.4349 KOps/s 81.0686 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[False-True-True-True-True] 61.6250μs 30.1759μs 33.1391 KOps/s 32.4256 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-True-True-True-False] 51.8370μs 18.7790μs 53.2508 KOps/s 51.4269 KOps/s $\color{#35bf28}+3.55\%$
test_step_mdp_speed[False-True-True-False-True] 45.4450μs 19.1213μs 52.2978 KOps/s 50.5195 KOps/s $\color{#35bf28}+3.52\%$
test_step_mdp_speed[False-True-True-False-False] 37.0590μs 11.6919μs 85.5295 KOps/s 83.9397 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-False-True-True] 67.7070μs 31.3355μs 31.9127 KOps/s 30.9568 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[False-True-False-True-False] 58.4090μs 20.2140μs 49.4707 KOps/s 47.3823 KOps/s $\color{#35bf28}+4.41\%$
test_step_mdp_speed[False-True-False-False-True] 2.9239ms 20.7092μs 48.2878 KOps/s 47.7150 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-False-False-False] 37.3100μs 13.2114μs 75.6924 KOps/s 73.9473 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-False-True-True-True] 75.6010μs 33.1355μs 30.1792 KOps/s 29.2671 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[False-False-True-True-False] 79.2550μs 21.8402μs 45.7871 KOps/s 44.7816 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-False-True-False-True] 49.9330μs 20.4802μs 48.8276 KOps/s 47.0094 KOps/s $\color{#35bf28}+3.87\%$
test_step_mdp_speed[False-False-True-False-False] 43.1810μs 13.1662μs 75.9518 KOps/s 72.9233 KOps/s $\color{#35bf28}+4.15\%$
test_step_mdp_speed[False-False-False-True-True] 74.5500μs 34.0180μs 29.3962 KOps/s 28.3241 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[False-False-False-True-False] 50.8050μs 22.9641μs 43.5462 KOps/s 41.7816 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[False-False-False-False-True] 60.8640μs 22.0228μs 45.4075 KOps/s 44.6016 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-False-False-False-False] 57.3510μs 14.4938μs 68.9949 KOps/s 66.9301 KOps/s $\color{#35bf28}+3.09\%$
test_values[generalized_advantage_estimate-True-True] 9.8273ms 9.3741ms 106.6769 Ops/s 104.4265 Ops/s $\color{#35bf28}+2.16\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.3032ms 33.4823ms 29.8665 Ops/s 30.0004 Ops/s $\color{#d91a1a}-0.45\%$
test_values[td0_return_estimate-False-False] 0.2268ms 0.1721ms 5.8101 KOps/s 5.9726 KOps/s $\color{#d91a1a}-2.72\%$
test_values[td1_return_estimate-False-False] 23.9256ms 23.4208ms 42.6972 Ops/s 41.1079 Ops/s $\color{#35bf28}+3.87\%$
test_values[vec_td1_return_estimate-False-False] 60.3284ms 34.5642ms 28.9317 Ops/s 29.9231 Ops/s $\color{#d91a1a}-3.31\%$
test_values[td_lambda_return_estimate-True-False] 35.7640ms 34.0178ms 29.3963 Ops/s 28.8929 Ops/s $\color{#35bf28}+1.74\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.1483ms 33.5158ms 29.8367 Ops/s 29.9258 Ops/s $\color{#d91a1a}-0.30\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.5396ms 8.2065ms 121.8541 Ops/s 117.8002 Ops/s $\color{#35bf28}+3.44\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.2493ms 2.0264ms 493.4770 Ops/s 511.4337 Ops/s $\color{#d91a1a}-3.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5716ms 0.3584ms 2.7898 KOps/s 2.7986 KOps/s $\color{#d91a1a}-0.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.5335ms 45.4473ms 22.0035 Ops/s 23.6757 Ops/s $\textbf{\color{#d91a1a}-7.06\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1332ms 3.0545ms 327.3831 Ops/s 328.1900 Ops/s $\color{#d91a1a}-0.25\%$
test_dqn_speed[False-None] 6.3358ms 1.3357ms 748.6664 Ops/s 751.6087 Ops/s $\color{#d91a1a}-0.39\%$
test_dqn_speed[False-backward] 1.8751ms 1.8239ms 548.2637 Ops/s 548.8286 Ops/s $\color{#d91a1a}-0.10\%$
test_dqn_speed[True-None] 0.6452ms 0.4598ms 2.1748 KOps/s 2.1444 KOps/s $\color{#35bf28}+1.42\%$
test_dqn_speed[True-backward] 0.9511ms 0.8872ms 1.1272 KOps/s 985.6774 Ops/s $\textbf{\color{#35bf28}+14.36\%}$
test_dqn_speed[reduce-overhead-None] 0.7213ms 0.4720ms 2.1187 KOps/s 2.1391 KOps/s $\color{#d91a1a}-0.95\%$
test_dqn_speed[reduce-overhead-backward] 0.9452ms 0.8882ms 1.1259 KOps/s 1.1305 KOps/s $\color{#d91a1a}-0.41\%$
test_ddpg_speed[False-None] 3.3797ms 2.7765ms 360.1670 Ops/s 360.1792 Ops/s $-0.00\%$
test_ddpg_speed[False-backward] 4.0680ms 3.9463ms 253.4016 Ops/s 256.1994 Ops/s $\color{#d91a1a}-1.09\%$
test_ddpg_speed[True-None] 1.2291ms 1.0118ms 988.3849 Ops/s 994.9587 Ops/s $\color{#d91a1a}-0.66\%$
test_ddpg_speed[True-backward] 2.3025ms 2.1682ms 461.2173 Ops/s 524.6125 Ops/s $\textbf{\color{#d91a1a}-12.08\%}$
test_ddpg_speed[reduce-overhead-None] 1.3498ms 1.0047ms 995.3288 Ops/s 992.7567 Ops/s $\color{#35bf28}+0.26\%$
test_ddpg_speed[reduce-overhead-backward] 2.1394ms 1.9192ms 521.0400 Ops/s 521.1188 Ops/s $\color{#d91a1a}-0.02\%$
test_sac_speed[False-None] 8.3827ms 7.9365ms 126.0008 Ops/s 126.4246 Ops/s $\color{#d91a1a}-0.34\%$
test_sac_speed[False-backward] 13.8589ms 10.7677ms 92.8700 Ops/s 93.7508 Ops/s $\color{#d91a1a}-0.94\%$
test_sac_speed[True-None] 2.4337ms 1.8312ms 546.0769 Ops/s 542.0375 Ops/s $\color{#35bf28}+0.75\%$
test_sac_speed[True-backward] 4.5061ms 3.5785ms 279.4477 Ops/s 281.4207 Ops/s $\color{#d91a1a}-0.70\%$
test_sac_speed[reduce-overhead-None] 2.1254ms 1.8309ms 546.1686 Ops/s 545.2931 Ops/s $\color{#35bf28}+0.16\%$
test_sac_speed[reduce-overhead-backward] 3.6746ms 3.5279ms 283.4572 Ops/s 280.5969 Ops/s $\color{#35bf28}+1.02\%$
test_redq_speed[False-None] 15.1179ms 12.7200ms 78.6161 Ops/s 78.7217 Ops/s $\color{#d91a1a}-0.13\%$
test_redq_speed[False-backward] 40.9736ms 22.8860ms 43.6948 Ops/s 44.8599 Ops/s $\color{#d91a1a}-2.60\%$
test_redq_speed[True-None] 6.1267ms 4.5970ms 217.5331 Ops/s 214.4237 Ops/s $\color{#35bf28}+1.45\%$
test_redq_speed[True-backward] 14.0909ms 12.5807ms 79.4866 Ops/s 81.7885 Ops/s $\color{#d91a1a}-2.81\%$
test_redq_speed[reduce-overhead-None] 5.4992ms 4.6293ms 216.0166 Ops/s 210.6916 Ops/s $\color{#35bf28}+2.53\%$
test_redq_speed[reduce-overhead-backward] 13.5775ms 12.1538ms 82.2786 Ops/s 81.6368 Ops/s $\color{#35bf28}+0.79\%$
test_redq_deprec_speed[False-None] 15.5250ms 12.6845ms 78.8364 Ops/s 76.0455 Ops/s $\color{#35bf28}+3.67\%$
test_redq_deprec_speed[False-backward] 20.7756ms 18.6763ms 53.5439 Ops/s 54.5239 Ops/s $\color{#d91a1a}-1.80\%$
test_redq_deprec_speed[True-None] 4.3992ms 3.6009ms 277.7113 Ops/s 276.3922 Ops/s $\color{#35bf28}+0.48\%$
test_redq_deprec_speed[True-backward] 9.0012ms 8.1149ms 123.2306 Ops/s 123.4166 Ops/s $\color{#d91a1a}-0.15\%$
test_redq_deprec_speed[reduce-overhead-None] 4.1861ms 3.5313ms 283.1823 Ops/s 275.4686 Ops/s $\color{#35bf28}+2.80\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.5913ms 7.9581ms 125.6582 Ops/s 122.1063 Ops/s $\color{#35bf28}+2.91\%$
test_td3_speed[False-None] 8.4546ms 7.6985ms 129.8949 Ops/s 130.0780 Ops/s $\color{#d91a1a}-0.14\%$
test_td3_speed[False-backward] 12.7264ms 10.3007ms 97.0812 Ops/s 98.8096 Ops/s $\color{#d91a1a}-1.75\%$
test_td3_speed[True-None] 1.9219ms 1.7095ms 584.9602 Ops/s 580.1743 Ops/s $\color{#35bf28}+0.82\%$
test_td3_speed[True-backward] 4.0555ms 3.3226ms 300.9664 Ops/s 299.0438 Ops/s $\color{#35bf28}+0.64\%$
test_td3_speed[reduce-overhead-None] 1.9185ms 1.7083ms 585.3623 Ops/s 577.9944 Ops/s $\color{#35bf28}+1.27\%$
test_td3_speed[reduce-overhead-backward] 3.3486ms 3.2806ms 304.8227 Ops/s 297.5971 Ops/s $\color{#35bf28}+2.43\%$
test_cql_speed[False-None] 36.5379ms 35.0111ms 28.5624 Ops/s 27.9565 Ops/s $\color{#35bf28}+2.17\%$
test_cql_speed[False-backward] 46.4069ms 44.8304ms 22.3063 Ops/s 21.1666 Ops/s $\textbf{\color{#35bf28}+5.38\%}$
test_cql_speed[True-None] 17.1901ms 15.6477ms 63.9071 Ops/s 64.0967 Ops/s $\color{#d91a1a}-0.30\%$
test_cql_speed[True-backward] 23.1096ms 22.1562ms 45.1341 Ops/s 44.6779 Ops/s $\color{#35bf28}+1.02\%$
test_cql_speed[reduce-overhead-None] 16.8022ms 15.4251ms 64.8295 Ops/s 62.4538 Ops/s $\color{#35bf28}+3.80\%$
test_cql_speed[reduce-overhead-backward] 23.7492ms 22.5456ms 44.3546 Ops/s 44.6524 Ops/s $\color{#d91a1a}-0.67\%$
test_a2c_speed[False-None] 8.7844ms 7.0814ms 141.2145 Ops/s 140.3469 Ops/s $\color{#35bf28}+0.62\%$
test_a2c_speed[False-backward] 15.0778ms 14.0340ms 71.2556 Ops/s 69.7563 Ops/s $\color{#35bf28}+2.15\%$
test_a2c_speed[True-None] 3.7320ms 3.3008ms 302.9540 Ops/s 295.3186 Ops/s $\color{#35bf28}+2.59\%$
test_a2c_speed[True-backward] 10.6772ms 9.8031ms 102.0082 Ops/s 100.6194 Ops/s $\color{#35bf28}+1.38\%$
test_a2c_speed[reduce-overhead-None] 3.9439ms 3.2987ms 303.1486 Ops/s 300.4168 Ops/s $\color{#35bf28}+0.91\%$
test_a2c_speed[reduce-overhead-backward] 10.3520ms 9.8131ms 101.9048 Ops/s 100.3011 Ops/s $\color{#35bf28}+1.60\%$
test_ppo_speed[False-None] 10.2442ms 7.4282ms 134.6225 Ops/s 132.2807 Ops/s $\color{#35bf28}+1.77\%$
test_ppo_speed[False-backward] 17.9058ms 14.7576ms 67.7618 Ops/s 65.4300 Ops/s $\color{#35bf28}+3.56\%$
test_ppo_speed[True-None] 3.9457ms 3.7049ms 269.9153 Ops/s 263.1933 Ops/s $\color{#35bf28}+2.55\%$
test_ppo_speed[True-backward] 11.9227ms 9.8287ms 101.7425 Ops/s 101.8807 Ops/s $\color{#d91a1a}-0.14\%$
test_ppo_speed[reduce-overhead-None] 4.3288ms 3.6720ms 272.3305 Ops/s 265.3307 Ops/s $\color{#35bf28}+2.64\%$
test_ppo_speed[reduce-overhead-backward] 10.6434ms 9.6677ms 103.4368 Ops/s 102.1813 Ops/s $\color{#35bf28}+1.23\%$
test_reinforce_speed[False-None] 7.2362ms 6.4312ms 155.4931 Ops/s 151.6748 Ops/s $\color{#35bf28}+2.52\%$
test_reinforce_speed[False-backward] 10.7167ms 9.8181ms 101.8529 Ops/s 101.2769 Ops/s $\color{#35bf28}+0.57\%$
test_reinforce_speed[True-None] 3.2260ms 2.6244ms 381.0378 Ops/s 362.2398 Ops/s $\textbf{\color{#35bf28}+5.19\%}$
test_reinforce_speed[True-backward] 8.9788ms 8.6713ms 115.3223 Ops/s 113.7429 Ops/s $\color{#35bf28}+1.39\%$
test_reinforce_speed[reduce-overhead-None] 3.6492ms 2.6387ms 378.9808 Ops/s 373.6365 Ops/s $\color{#35bf28}+1.43\%$
test_reinforce_speed[reduce-overhead-backward] 8.9508ms 8.5741ms 116.6299 Ops/s 113.4767 Ops/s $\color{#35bf28}+2.78\%$
test_iql_speed[False-None] 33.7206ms 31.6586ms 31.5869 Ops/s 30.9758 Ops/s $\color{#35bf28}+1.97\%$
test_iql_speed[False-backward] 46.1533ms 44.7097ms 22.3665 Ops/s 21.7929 Ops/s $\color{#35bf28}+2.63\%$
test_iql_speed[True-None] 11.4035ms 10.5285ms 94.9801 Ops/s 93.3917 Ops/s $\color{#35bf28}+1.70\%$
test_iql_speed[True-backward] 24.3568ms 21.7967ms 45.8784 Ops/s 45.0404 Ops/s $\color{#35bf28}+1.86\%$
test_iql_speed[reduce-overhead-None] 11.0009ms 10.5252ms 95.0105 Ops/s 90.9814 Ops/s $\color{#35bf28}+4.43\%$
test_iql_speed[reduce-overhead-backward] 22.5146ms 21.5815ms 46.3359 Ops/s 45.6866 Ops/s $\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9519ms 4.7602ms 210.0761 Ops/s 206.1267 Ops/s $\color{#35bf28}+1.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.3689ms 0.5935ms 1.6849 KOps/s 1.5122 KOps/s $\textbf{\color{#35bf28}+11.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8493ms 0.6078ms 1.6454 KOps/s 1.5942 KOps/s $\color{#35bf28}+3.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1364ms 4.5899ms 217.8678 Ops/s 214.0254 Ops/s $\color{#35bf28}+1.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9059ms 0.6473ms 1.5449 KOps/s 1.5715 KOps/s $\color{#d91a1a}-1.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8287ms 0.6111ms 1.6364 KOps/s 1.6040 KOps/s $\color{#35bf28}+2.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1346ms 1.8753ms 533.2344 Ops/s 529.1930 Ops/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 6.3686ms 1.9170ms 521.6408 Ops/s 550.0689 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1680ms 4.7098ms 212.3253 Ops/s 208.9812 Ops/s $\color{#35bf28}+1.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.5155ms 0.7813ms 1.2800 KOps/s 541.5320 Ops/s $\textbf{\color{#35bf28}+136.36\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9469ms 0.7537ms 1.3269 KOps/s 1.3014 KOps/s $\color{#35bf28}+1.96\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9522ms 4.5275ms 220.8724 Ops/s 212.8820 Ops/s $\color{#35bf28}+3.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.2035ms 0.5570ms 1.7952 KOps/s 1.5093 KOps/s $\textbf{\color{#35bf28}+18.94\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8196ms 0.6086ms 1.6432 KOps/s 1.5812 KOps/s $\color{#35bf28}+3.93\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0566ms 4.5595ms 219.3212 Ops/s 212.7799 Ops/s $\color{#35bf28}+3.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1341ms 0.5985ms 1.6709 KOps/s 1.5411 KOps/s $\textbf{\color{#35bf28}+8.42\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.0127ms 0.6213ms 1.6095 KOps/s 1.6234 KOps/s $\color{#d91a1a}-0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.5864ms 4.7243ms 211.6697 Ops/s 205.9000 Ops/s $\color{#35bf28}+2.80\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0271ms 0.7282ms 1.3733 KOps/s 1.2547 KOps/s $\textbf{\color{#35bf28}+9.44\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.4026ms 0.7676ms 1.3027 KOps/s 1.2897 KOps/s $\color{#35bf28}+1.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3885s 11.8645ms 84.2852 Ops/s 235.8372 Ops/s $\textbf{\color{#d91a1a}-64.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.2397ms 2.3324ms 428.7511 Ops/s 447.4860 Ops/s $\color{#d91a1a}-4.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.7396ms 1.2527ms 798.2869 Ops/s 725.8475 Ops/s $\textbf{\color{#35bf28}+9.98\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.5534ms 4.1940ms 238.4375 Ops/s 236.6211 Ops/s $\color{#35bf28}+0.77\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.0038ms 2.2930ms 436.1154 Ops/s 425.2829 Ops/s $\color{#35bf28}+2.55\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.1242ms 1.2159ms 822.4132 Ops/s 794.0682 Ops/s $\color{#35bf28}+3.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3673s 11.5950ms 86.2441 Ops/s 237.1212 Ops/s $\textbf{\color{#d91a1a}-63.63\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 3.5679ms 2.3043ms 433.9637 Ops/s 414.5802 Ops/s $\color{#35bf28}+4.68\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8271ms 1.3291ms 752.3609 Ops/s 582.9353 Ops/s $\textbf{\color{#35bf28}+29.06\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 11.4997ms 11.0905ms 90.1670 Ops/s 85.9117 Ops/s $\color{#35bf28}+4.95\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.6620ms 14.2774ms 70.0407 Ops/s 66.2327 Ops/s $\textbf{\color{#35bf28}+5.75\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 20.7617ms 19.8733ms 50.3188 Ops/s 48.1746 Ops/s $\color{#35bf28}+4.45\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.2746ms 14.3826ms 69.5285 Ops/s 67.2015 Ops/s $\color{#35bf28}+3.46\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 20.2307ms 19.7983ms 50.5094 Ops/s 48.6992 Ops/s $\color{#35bf28}+3.72\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 16.3947ms 15.5159ms 64.4500 Ops/s 61.6873 Ops/s $\color{#35bf28}+4.48\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 4, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}29$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7760s 0.7629s 1.3108 Ops/s 1.3419 Ops/s $\color{#d91a1a}-2.32\%$
test_transformed 1.0857s 1.0170s 0.9833 Ops/s 1.0130 Ops/s $\color{#d91a1a}-2.93\%$
test_serial 2.2886s 2.1592s 0.4631 Ops/s 0.4639 Ops/s $\color{#d91a1a}-0.17\%$
test_parallel 2.1641s 2.0930s 0.4778 Ops/s 0.5117 Ops/s $\textbf{\color{#d91a1a}-6.63\%}$
test_step_mdp_speed[True-True-True-True-True] 0.1752ms 35.1453μs 28.4533 KOps/s 26.7520 KOps/s $\textbf{\color{#35bf28}+6.36\%}$
test_step_mdp_speed[True-True-True-True-False] 49.9510μs 20.5832μs 48.5834 KOps/s 46.6461 KOps/s $\color{#35bf28}+4.15\%$
test_step_mdp_speed[True-True-True-False-True] 49.1410μs 20.0544μs 49.8644 KOps/s 46.5001 KOps/s $\textbf{\color{#35bf28}+7.23\%}$
test_step_mdp_speed[True-True-True-False-False] 38.0900μs 11.6353μs 85.9456 KOps/s 81.0606 KOps/s $\textbf{\color{#35bf28}+6.03\%}$
test_step_mdp_speed[True-True-False-True-True] 68.4610μs 37.8803μs 26.3989 KOps/s 25.2257 KOps/s $\color{#35bf28}+4.65\%$
test_step_mdp_speed[True-True-False-True-False] 47.1000μs 22.3653μs 44.7120 KOps/s 43.0312 KOps/s $\color{#35bf28}+3.91\%$
test_step_mdp_speed[True-True-False-False-True] 63.8810μs 21.7922μs 45.8880 KOps/s 41.5805 KOps/s $\textbf{\color{#35bf28}+10.36\%}$
test_step_mdp_speed[True-True-False-False-False] 42.7300μs 13.5879μs 73.5950 KOps/s 68.7454 KOps/s $\textbf{\color{#35bf28}+7.05\%}$
test_step_mdp_speed[True-False-True-True-True] 76.1420μs 39.6140μs 25.2436 KOps/s 23.8803 KOps/s $\textbf{\color{#35bf28}+5.71\%}$
test_step_mdp_speed[True-False-True-True-False] 58.2810μs 24.1327μs 41.4375 KOps/s 38.6289 KOps/s $\textbf{\color{#35bf28}+7.27\%}$
test_step_mdp_speed[True-False-True-False-True] 58.9610μs 21.6235μs 46.2460 KOps/s 41.6404 KOps/s $\textbf{\color{#35bf28}+11.06\%}$
test_step_mdp_speed[True-False-True-False-False] 55.3410μs 13.6379μs 73.3248 KOps/s 69.9474 KOps/s $\color{#35bf28}+4.83\%$
test_step_mdp_speed[True-False-False-True-True] 78.8920μs 41.7611μs 23.9457 KOps/s 22.8551 KOps/s $\color{#35bf28}+4.77\%$
test_step_mdp_speed[True-False-False-True-False] 67.5210μs 26.3478μs 37.9538 KOps/s 35.8134 KOps/s $\textbf{\color{#35bf28}+5.98\%}$
test_step_mdp_speed[True-False-False-False-True] 51.0610μs 23.9812μs 41.6992 KOps/s 40.0226 KOps/s $\color{#35bf28}+4.19\%$
test_step_mdp_speed[True-False-False-False-False] 82.5310μs 15.6989μs 63.6986 KOps/s 61.3407 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[False-True-True-True-True] 73.1310μs 40.2420μs 24.8497 KOps/s 23.6564 KOps/s $\textbf{\color{#35bf28}+5.04\%}$
test_step_mdp_speed[False-True-True-True-False] 47.1110μs 25.1232μs 39.8039 KOps/s 39.0991 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-True-True-False-True] 53.1710μs 25.9461μs 38.5415 KOps/s 36.9305 KOps/s $\color{#35bf28}+4.36\%$
test_step_mdp_speed[False-True-True-False-False] 44.1610μs 15.7581μs 63.4596 KOps/s 62.1720 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[False-True-False-True-True] 73.2910μs 42.1117μs 23.7464 KOps/s 22.8645 KOps/s $\color{#35bf28}+3.86\%$
test_step_mdp_speed[False-True-False-True-False] 54.5810μs 26.1948μs 38.1756 KOps/s 36.5450 KOps/s $\color{#35bf28}+4.46\%$
test_step_mdp_speed[False-True-False-False-True] 3.3765ms 28.0205μs 35.6882 KOps/s 35.1412 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-False-False-False] 43.2610μs 17.6432μs 56.6790 KOps/s 55.4058 KOps/s $\color{#35bf28}+2.30\%$
test_step_mdp_speed[False-False-True-True-True] 70.1810μs 44.5405μs 22.4515 KOps/s 21.7249 KOps/s $\color{#35bf28}+3.34\%$
test_step_mdp_speed[False-False-True-True-False] 64.6910μs 28.3784μs 35.2381 KOps/s 33.9173 KOps/s $\color{#35bf28}+3.89\%$
test_step_mdp_speed[False-False-True-False-True] 62.5810μs 26.7795μs 37.3420 KOps/s 35.4887 KOps/s $\textbf{\color{#35bf28}+5.22\%}$
test_step_mdp_speed[False-False-True-False-False] 51.3610μs 17.2505μs 57.9694 KOps/s 56.0297 KOps/s $\color{#35bf28}+3.46\%$
test_step_mdp_speed[False-False-False-True-True] 76.7010μs 44.5385μs 22.4525 KOps/s 21.3111 KOps/s $\textbf{\color{#35bf28}+5.36\%}$
test_step_mdp_speed[False-False-False-True-False] 62.4510μs 30.3985μs 32.8963 KOps/s 31.3741 KOps/s $\color{#35bf28}+4.85\%$
test_step_mdp_speed[False-False-False-False-True] 62.0910μs 28.9173μs 34.5814 KOps/s 33.2287 KOps/s $\color{#35bf28}+4.07\%$
test_step_mdp_speed[False-False-False-False-False] 43.3610μs 19.0461μs 52.5042 KOps/s 50.2865 KOps/s $\color{#35bf28}+4.41\%$
test_values[generalized_advantage_estimate-True-True] 25.0636ms 24.5353ms 40.7575 Ops/s 41.2806 Ops/s $\color{#d91a1a}-1.27\%$
test_values[vec_generalized_advantage_estimate-True-True] 93.0412ms 2.7508ms 363.5297 Ops/s 330.8364 Ops/s $\textbf{\color{#35bf28}+9.88\%}$
test_values[td0_return_estimate-False-False] 85.7820μs 66.8850μs 14.9510 KOps/s 15.0511 KOps/s $\color{#d91a1a}-0.66\%$
test_values[td1_return_estimate-False-False] 55.2916ms 55.0657ms 18.1601 Ops/s 18.4319 Ops/s $\color{#d91a1a}-1.47\%$
test_values[vec_td1_return_estimate-False-False] 1.3984ms 1.0780ms 927.6812 Ops/s 929.3211 Ops/s $\color{#d91a1a}-0.18\%$
test_values[td_lambda_return_estimate-True-False] 88.5991ms 87.2963ms 11.4552 Ops/s 11.5589 Ops/s $\color{#d91a1a}-0.90\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3960ms 1.0747ms 930.4817 Ops/s 922.5744 Ops/s $\color{#35bf28}+0.86\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.7522ms 24.2944ms 41.1617 Ops/s 41.5614 Ops/s $\color{#d91a1a}-0.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.2167ms 0.7553ms 1.3239 KOps/s 1.3470 KOps/s $\color{#d91a1a}-1.72\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7622ms 0.6611ms 1.5127 KOps/s 1.5195 KOps/s $\color{#d91a1a}-0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5495ms 1.4709ms 679.8367 Ops/s 678.2938 Ops/s $\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7244ms 0.6766ms 1.4779 KOps/s 1.4890 KOps/s $\color{#d91a1a}-0.74\%$
test_dqn_speed[False-None] 6.8030ms 1.3060ms 765.7008 Ops/s 769.1968 Ops/s $\color{#d91a1a}-0.45\%$
test_dqn_speed[False-backward] 1.8818ms 1.8113ms 552.0891 Ops/s 544.7726 Ops/s $\color{#35bf28}+1.34\%$
test_dqn_speed[True-None] 0.6835ms 0.5469ms 1.8283 KOps/s 1.7313 KOps/s $\textbf{\color{#35bf28}+5.60\%}$
test_dqn_speed[True-backward] 1.0400ms 0.9976ms 1.0024 KOps/s 982.8745 Ops/s $\color{#35bf28}+1.99\%$
test_dqn_speed[reduce-overhead-None] 0.6678ms 0.5474ms 1.8268 KOps/s 1.7754 KOps/s $\color{#35bf28}+2.90\%$
test_dqn_speed[reduce-overhead-backward] 1.0354ms 1.0031ms 996.8892 Ops/s 1.0057 KOps/s $\color{#d91a1a}-0.87\%$
test_ddpg_speed[False-None] 3.0675ms 2.6755ms 373.7681 Ops/s 376.3410 Ops/s $\color{#d91a1a}-0.68\%$
test_ddpg_speed[False-backward] 3.9564ms 3.8276ms 261.2587 Ops/s 259.1721 Ops/s $\color{#35bf28}+0.81\%$
test_ddpg_speed[True-None] 1.3554ms 1.2259ms 815.7024 Ops/s 815.1250 Ops/s $\color{#35bf28}+0.07\%$
test_ddpg_speed[True-backward] 2.4588ms 2.2517ms 444.1011 Ops/s 426.4700 Ops/s $\color{#35bf28}+4.13\%$
test_ddpg_speed[reduce-overhead-None] 1.3710ms 1.2273ms 814.7724 Ops/s 818.6965 Ops/s $\color{#d91a1a}-0.48\%$
test_ddpg_speed[reduce-overhead-backward] 2.2445ms 2.1961ms 455.3561 Ops/s 456.4054 Ops/s $\color{#d91a1a}-0.23\%$
test_sac_speed[False-None] 8.5790ms 7.4609ms 134.0315 Ops/s 133.2527 Ops/s $\color{#35bf28}+0.58\%$
test_sac_speed[False-backward] 10.8218ms 10.5540ms 94.7505 Ops/s 94.0089 Ops/s $\color{#35bf28}+0.79\%$
test_sac_speed[True-None] 2.3537ms 1.9823ms 504.4765 Ops/s 489.7646 Ops/s $\color{#35bf28}+3.00\%$
test_sac_speed[True-backward] 3.9973ms 3.8649ms 258.7361 Ops/s 219.4428 Ops/s $\textbf{\color{#35bf28}+17.91\%}$
test_sac_speed[reduce-overhead-None] 2.3756ms 1.9876ms 503.1104 Ops/s 502.3962 Ops/s $\color{#35bf28}+0.14\%$
test_sac_speed[reduce-overhead-backward] 4.0864ms 3.8980ms 256.5394 Ops/s 258.3508 Ops/s $\color{#d91a1a}-0.70\%$
test_redq_speed[False-None] 16.0655ms 11.4867ms 87.0573 Ops/s 98.7633 Ops/s $\textbf{\color{#d91a1a}-11.85\%}$
test_redq_speed[False-backward] 18.0015ms 17.1979ms 58.1466 Ops/s 56.7374 Ops/s $\color{#35bf28}+2.48\%$
test_redq_speed[True-None] 3.7916ms 3.4919ms 286.3793 Ops/s 280.2242 Ops/s $\color{#35bf28}+2.20\%$
test_redq_speed[True-backward] 8.9436ms 8.5579ms 116.8504 Ops/s 109.9974 Ops/s $\textbf{\color{#35bf28}+6.23\%}$
test_redq_speed[reduce-overhead-None] 3.9852ms 3.5186ms 284.2032 Ops/s 295.2866 Ops/s $\color{#d91a1a}-3.75\%$
test_redq_speed[reduce-overhead-backward] 8.8244ms 8.4514ms 118.3236 Ops/s 118.5072 Ops/s $\color{#d91a1a}-0.15\%$
test_redq_deprec_speed[False-None] 11.3135ms 10.5417ms 94.8613 Ops/s 93.7481 Ops/s $\color{#35bf28}+1.19\%$
test_redq_deprec_speed[False-backward] 15.8922ms 15.2735ms 65.4728 Ops/s 65.4837 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_deprec_speed[True-None] 3.6992ms 3.2154ms 311.0028 Ops/s 307.3968 Ops/s $\color{#35bf28}+1.17\%$
test_redq_deprec_speed[True-backward] 7.4181ms 7.0946ms 140.9531 Ops/s 146.4769 Ops/s $\color{#d91a1a}-3.77\%$
test_redq_deprec_speed[reduce-overhead-None] 3.6791ms 3.2096ms 311.5693 Ops/s 317.0675 Ops/s $\color{#d91a1a}-1.73\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.6067ms 7.1234ms 140.3815 Ops/s 139.0505 Ops/s $\color{#35bf28}+0.96\%$
test_td3_speed[False-None] 7.5997ms 7.3755ms 135.5847 Ops/s 133.1425 Ops/s $\color{#35bf28}+1.83\%$
test_td3_speed[False-backward] 10.6618ms 10.1801ms 98.2311 Ops/s 96.9410 Ops/s $\color{#35bf28}+1.33\%$
test_td3_speed[True-None] 1.9336ms 1.8864ms 530.1088 Ops/s 530.3860 Ops/s $\color{#d91a1a}-0.05\%$
test_td3_speed[True-backward] 3.8158ms 3.6448ms 274.3663 Ops/s 274.1870 Ops/s $\color{#35bf28}+0.07\%$
test_td3_speed[reduce-overhead-None] 1.9899ms 1.8768ms 532.8109 Ops/s 533.7526 Ops/s $\color{#d91a1a}-0.18\%$
test_td3_speed[reduce-overhead-backward] 3.7578ms 3.6752ms 272.0953 Ops/s 280.5878 Ops/s $\color{#d91a1a}-3.03\%$
test_cql_speed[False-None] 28.6344ms 24.7349ms 40.4287 Ops/s 41.1314 Ops/s $\color{#d91a1a}-1.71\%$
test_cql_speed[False-backward] 39.4692ms 35.0942ms 28.4948 Ops/s 30.2152 Ops/s $\textbf{\color{#d91a1a}-5.69\%}$
test_cql_speed[True-None] 11.2714ms 10.8065ms 92.5370 Ops/s 93.9855 Ops/s $\color{#d91a1a}-1.54\%$
test_cql_speed[True-backward] 16.8766ms 16.5722ms 60.3419 Ops/s 60.8507 Ops/s $\color{#d91a1a}-0.84\%$
test_cql_speed[reduce-overhead-None] 11.4802ms 10.8071ms 92.5322 Ops/s 92.5965 Ops/s $\color{#d91a1a}-0.07\%$
test_cql_speed[reduce-overhead-backward] 17.3117ms 16.5338ms 60.4820 Ops/s 61.0623 Ops/s $\color{#d91a1a}-0.95\%$
test_a2c_speed[False-None] 5.6052ms 5.2163ms 191.7068 Ops/s 185.9726 Ops/s $\color{#35bf28}+3.08\%$
test_a2c_speed[False-backward] 11.8900ms 11.5588ms 86.5143 Ops/s 85.5532 Ops/s $\color{#35bf28}+1.12\%$
test_a2c_speed[True-None] 3.2423ms 3.0051ms 332.7627 Ops/s 328.1339 Ops/s $\color{#35bf28}+1.41\%$
test_a2c_speed[True-backward] 8.8808ms 8.3532ms 119.7142 Ops/s 115.1226 Ops/s $\color{#35bf28}+3.99\%$
test_a2c_speed[reduce-overhead-None] 3.1550ms 2.9882ms 334.6516 Ops/s 329.5286 Ops/s $\color{#35bf28}+1.55\%$
test_a2c_speed[reduce-overhead-backward] 8.8479ms 8.3450ms 119.8328 Ops/s 120.2982 Ops/s $\color{#d91a1a}-0.39\%$
test_ppo_speed[False-None] 6.2222ms 5.6261ms 177.7445 Ops/s 176.5716 Ops/s $\color{#35bf28}+0.66\%$
test_ppo_speed[False-backward] 13.7940ms 12.2066ms 81.9229 Ops/s 82.8949 Ops/s $\color{#d91a1a}-1.17\%$
test_ppo_speed[True-None] 3.5051ms 3.3577ms 297.8261 Ops/s 292.5105 Ops/s $\color{#35bf28}+1.82\%$
test_ppo_speed[True-backward] 8.9476ms 8.2064ms 121.8558 Ops/s 124.0590 Ops/s $\color{#d91a1a}-1.78\%$
test_ppo_speed[reduce-overhead-None] 3.7338ms 3.3748ms 296.3154 Ops/s 298.2176 Ops/s $\color{#d91a1a}-0.64\%$
test_ppo_speed[reduce-overhead-backward] 8.6610ms 8.0666ms 123.9686 Ops/s 123.2604 Ops/s $\color{#35bf28}+0.57\%$
test_reinforce_speed[False-None] 4.9170ms 4.3884ms 227.8742 Ops/s 225.8458 Ops/s $\color{#35bf28}+0.90\%$
test_reinforce_speed[False-backward] 7.3577ms 7.1275ms 140.3023 Ops/s 138.4607 Ops/s $\color{#35bf28}+1.33\%$
test_reinforce_speed[True-None] 2.3974ms 2.2071ms 453.0790 Ops/s 443.4667 Ops/s $\color{#35bf28}+2.17\%$
test_reinforce_speed[True-backward] 7.5458ms 6.9691ms 143.4912 Ops/s 126.6179 Ops/s $\textbf{\color{#35bf28}+13.33\%}$
test_reinforce_speed[reduce-overhead-None] 2.6176ms 2.1893ms 456.7612 Ops/s 456.5176 Ops/s $\color{#35bf28}+0.05\%$
test_reinforce_speed[reduce-overhead-backward] 7.1497ms 6.9084ms 144.7518 Ops/s 142.7335 Ops/s $\color{#35bf28}+1.41\%$
test_iql_speed[False-None] 20.3077ms 19.3201ms 51.7595 Ops/s 50.5605 Ops/s $\color{#35bf28}+2.37\%$
test_iql_speed[False-backward] 30.6060ms 29.7959ms 33.5617 Ops/s 32.4728 Ops/s $\color{#35bf28}+3.35\%$
test_iql_speed[True-None] 7.3256ms 6.6822ms 149.6519 Ops/s 152.3924 Ops/s $\color{#d91a1a}-1.80\%$
test_iql_speed[True-backward] 16.2109ms 15.2862ms 65.4186 Ops/s 65.8585 Ops/s $\color{#d91a1a}-0.67\%$
test_iql_speed[reduce-overhead-None] 7.1633ms 6.6371ms 150.6685 Ops/s 151.3657 Ops/s $\color{#d91a1a}-0.46\%$
test_iql_speed[reduce-overhead-backward] 16.2458ms 15.2063ms 65.7624 Ops/s 65.7714 Ops/s $\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.4616ms 6.2875ms 159.0459 Ops/s 158.7056 Ops/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.8957ms 0.2702ms 3.7009 KOps/s 3.2853 KOps/s $\textbf{\color{#35bf28}+12.65\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4901ms 0.2910ms 3.4362 KOps/s 3.1555 KOps/s $\textbf{\color{#35bf28}+8.90\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3260ms 6.0558ms 165.1318 Ops/s 165.4333 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6199ms 0.2564ms 3.9003 KOps/s 3.3073 KOps/s $\textbf{\color{#35bf28}+17.93\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5632ms 0.3139ms 3.1854 KOps/s 3.6690 KOps/s $\textbf{\color{#d91a1a}-13.18\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5386ms 1.2086ms 827.4155 Ops/s 797.6365 Ops/s $\color{#35bf28}+3.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5623ms 1.1555ms 865.4263 Ops/s 820.7665 Ops/s $\textbf{\color{#35bf28}+5.44\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2711ms 6.1662ms 162.1754 Ops/s 159.1520 Ops/s $\color{#35bf28}+1.90\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.6685ms 0.4027ms 2.4830 KOps/s 2.1946 KOps/s $\textbf{\color{#35bf28}+13.14\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7161ms 0.3833ms 2.6088 KOps/s 2.3044 KOps/s $\textbf{\color{#35bf28}+13.21\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2636ms 6.1027ms 163.8624 Ops/s 164.5650 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6342ms 0.2704ms 3.6984 KOps/s 3.1318 KOps/s $\textbf{\color{#35bf28}+18.09\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4497ms 0.2486ms 4.0221 KOps/s 3.9411 KOps/s $\color{#35bf28}+2.06\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2640ms 6.0397ms 165.5705 Ops/s 165.7390 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6393ms 0.2576ms 3.8813 KOps/s 3.8364 KOps/s $\color{#35bf28}+1.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6481ms 0.2373ms 4.2149 KOps/s 4.2413 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4516ms 6.2172ms 160.8445 Ops/s 161.6701 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1261ms 0.4117ms 2.4292 KOps/s 2.1579 KOps/s $\textbf{\color{#35bf28}+12.57\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5883ms 0.3865ms 2.5875 KOps/s 2.1287 KOps/s $\textbf{\color{#35bf28}+21.56\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4144s 13.3799ms 74.7387 Ops/s 186.2290 Ops/s $\textbf{\color{#d91a1a}-59.87\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.6106ms 1.9767ms 505.9008 Ops/s 447.5180 Ops/s $\textbf{\color{#35bf28}+13.05\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.5913ms 1.2114ms 825.4671 Ops/s 957.3813 Ops/s $\textbf{\color{#d91a1a}-13.78\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.1705ms 5.2176ms 191.6592 Ops/s 34.2604 Ops/s $\textbf{\color{#35bf28}+459.42\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.8865ms 2.0320ms 492.1280 Ops/s 619.6989 Ops/s $\textbf{\color{#d91a1a}-20.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.1161ms 1.1662ms 857.5204 Ops/s 940.9679 Ops/s $\textbf{\color{#d91a1a}-8.87\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3696s 12.6938ms 78.7788 Ops/s 174.8767 Ops/s $\textbf{\color{#d91a1a}-54.95\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.9135ms 2.0981ms 476.6295 Ops/s 457.4617 Ops/s $\color{#35bf28}+4.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.3128ms 1.2139ms 823.7968 Ops/s 708.0797 Ops/s $\textbf{\color{#35bf28}+16.34\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.6786ms 12.6990ms 78.7463 Ops/s 77.0221 Ops/s $\color{#35bf28}+2.24\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 17.6603ms 16.4714ms 60.7113 Ops/s 60.6618 Ops/s $\color{#35bf28}+0.08\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 18.1550ms 17.5082ms 57.1161 Ops/s 56.6043 Ops/s $\color{#35bf28}+0.90\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 17.2277ms 16.5632ms 60.3750 Ops/s 60.7412 Ops/s $\color{#d91a1a}-0.60\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 17.5359ms 17.2128ms 58.0964 Ops/s 56.9197 Ops/s $\color{#35bf28}+2.07\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.7606ms 18.1171ms 55.1964 Ops/s 56.0565 Ops/s $\color{#d91a1a}-1.53\%$

@vmoens
Update
6578211 
[ghstack-poisoned]
@vmoens
Update
34cfaae 
[ghstack-poisoned]
@vmoens
Update
570afff 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 8, 2024
@vmoens
[Feature] MCTSForest
3e445fb 
ghstack-source-id: 9ac5cd3de39a4dbe1c7c33cb71ff6f45a886ae65
Pull Request resolved: #2307
@vmoens vmoens merged commit 570afff into gh/vmoens/4/base Nov 8, 2024
48 of 59 checks passed
vmoens added a commit that referenced this pull request Nov 8, 2024
@vmoens
[Feature] MCTSForest
e9d1677 
ghstack-source-id: 9ac5cd3de39a4dbe1c7c33cb71ff6f45a886ae65
Pull Request resolved: #2307
@vmoens vmoens deleted the gh/vmoens/4/head branch November 8, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot