Why is the default use_reentrant=True when no kwargs were set?

[simon-lund]

transformers/src/transformers/modeling_utils.py

Lines 2110 to 2111 in 350c5d1

	if gradient_checkpointing_kwargs is None:
	gradient_checkpointing_kwargs = {"use_reentrant": True}

Here use_reentrant=True is set when no kwargs are provided. From the docs of pytorch this seems to be the legacy variant.

Does this have any performance or other advantages that I am not aware of?

[amyeroberts]

Hi @simon-lund, thanks for opening an issue!

You can check the git blame to find the PRs which add certain lines - which should normally give you the reasons for the code logic.

In this case, this line was added in #28538 - according to the PR it's for preparation of an upcoming torch release, where it's necessary for us to explicitly pass the use_reentrant kwarg.

The value is set to true based on the current default PT behaviour c.f. this comment

[simon-lund]

Ah. I'm sorry, I only searched the issues and found several discussion about this, but I didn't search through the PRs
Thank you very much for the link as well as the hint with the PRs (I was not aware of that) 👍