-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Distributed Transactions (MSDTC) code blocking on disposal of Transaction Scope #76010
Comments
@nathangreaves thanks for testing out the rc, and thanks for the detailed report! I'll investigate this in the next few days. |
* Retake lock when using a dependent transaction from a TransactionScope (dotnet#76010). * Reset TransactionTransmitter and Receiver before reusing them (dotnet#76010). * Increase MSDTC startup timeout from 2.5 to 30 seconds (dotnet#75822) Fixes dotnet#76010 Fixes dotnet#75822
@nathangreaves thanks again for reporting this - a fix has been merged and is being backported for release for 7.0. It would be very useful if you could retry with a daily build once #76425 is merged, to make sure that everything works well for you (let me know if you have technical difficulties getting the daily build etc.). If you run into any further bugs, I'll prioritize fixing them for the 7.0 release. |
FYI note #76376, which will also get merged very soon and will require you to explicitly opt in in order to start a distributed transaction. |
* Distributed transaction fixes * Retake lock when using a dependent transaction from a TransactionScope (#76010). * Reset TransactionTransmitter and Receiver before reusing them (#76010). * Increase MSDTC startup timeout from 2.5 to 30 seconds (#75822) Fixes #76010 Fixes #75822 * Fix member naming Co-authored-by: Shay Rojansky <roji@roji.org>
Hi @roji |
@nathangreaves thanks for trying this.
Just to be on the safe side, are you saying you're no longer seeing the SynchronizationLockException, but only seeing blocking? I suspect this may be because of the following notes on this page:
But let me follow up internally on this and see how to make sure you have the latest build. |
I looked into this, and the latest rtm daily build (7.0.0-rtm.22478.9) corresponds to commit 3fed4a3, which does not yet contain the commit; the next daily build to appear should contain it. I'll check and update this issue once a build is available which contains the fix. |
I'm pretty sure that was the case but I no longer have .NET 7 installed on my machine so can't confirm right now.
That probably explains it then, thanks for checking! I'll hopefully have some time next week to try out another go at a .NET 7 build if it's available by then. |
@nathangreaves I've tried again today, and could indeed reproduce the hang with a newer daily build, which I confirmed contained the fix (the SynchronizationLockException was indeed gone thanks to the fix). To make a long story short, I've tracked the source of the problem to what looks like a deadlock in SqlClient - see dotnet/SqlClient#1800. As far as I can tell, the deadlock should be possible in .NET Framework as well, although I can't see it happening there. Another interesting fact is that when I reference a locally-built System.Transactions DLL directly from your project, that also doesn't repro - even if using the official runtime built from the same runtime commit reproduces the hang very reliably (this is why I didn't see the hang when originally working on your repro). This is most likely because of the timing-sensitive nature of the deadlock. |
FYI see dotnet/SqlClient#1800 (comment) for some more investigation. Specifically, it's possible to work around this bug by ensuring the transaction is promoted before the first SqlClient escalation. In other words, inserting this line before doing any SqlClient operations makes it go away: _ = TransactionInterop.GetTransmitterPropagationToken(transaction);
// If using an ambient transaction via TransactionScope:
_ = TransactionInterop.GetTransmitterPropagationToken(Transaction.Current!); |
@roji thanks for this! Do you think it's likely a fix will make it into the final build for .net 7? |
@nathangreaves the issue isn't on the .NET side, it's on the SqlClient side. I'll be in touch with them and do my best to make sure a fix is merged as soon as possible. Though note that you can at least work around this as above, in the meantime. |
@nathangreaves and others, I've tested this with the new Microsoft.Data.SqlClient 5.1.0 that came out today, and I can confirm that the deadlock is gone. Please give this a try and report how it goes! |
Description
When disposing of a TransactionScope object in .NET 7 code - where the transaction has been promoted to a distributed transaction and work has taken place in another process - the code blocks indefinitely.
Reproduction Steps
I encountered this when converting one of our .NET 4.8 ASP.NET web api applications to .NET 7, but I have written a simple console app that demonstrates this blocking behaviour - https://github.com/nathangreaves/NET7RC1-MSDTC-TestApp.
It contains 2 projects that have virtually the same code. One is targeting .NET 7 RC1, the other is .NET Framework 4.8.
The code starts a new TransactionScope, inserts a value into a database table, and writes the transmitter propagation token to a file on disk. When running the application a second time as a new instance, it will read this file, create a new TransactionScope (using the Transaction from the transmitter propogation token), and insert a value into a database table. It deletes the file following completion of the transaction. The first instance will then complete the outer transaction, and then repeat the whole process a second time.
The code in .NET Framework 4.8 works correctly, committing both transactions and continuing execution.
However the .NET 7 code blocks on disposing the outer transaction. Also the Inner Transaction throws the following exception which may be relevant:
However, if you run the applications in the following order:
The inner transaction completes successfully with the above exception, and the outer transaction (.NET 7) still blocks.
I attached the debugger to the .NET 7 project whilst it was blocked:
This screenshot shows the code has blocked waiting for a lock in transactionScope.Dispose();
The following screenshots show the output of the console applications for all of the scenarios in terms of which version of .NET is running for the 2 instances of the console application:
Outer Transaction .NET 7, Inner Transaction .NET 7 - outer transaction blocks on dispose, inner transaction throws SynchronizationLockException exception:
Outer Transaction .NET 7, Inner Transaction .NET 4.8 - outer transaction blocks on dispose, inner transaction successful:
Outer Transaction .NET 4.8, Inner Transaction .NET 7 - outer transactions complete, inner transactions throw SynchronizationLockException on dispose (but still complete):
Outer Transaction .NET 4.8, Inner Transaction .NET 4.8 - outer and inner transactions complete and execution continues
Expected behavior
Execution does not block on transactionScope.Dispose() when completing a distributed transaction.
Execution does not throw exception System.Threading.SynchronizationLockException when completing a distributed transaction.
Actual behavior
When disposing of a TransactionScope object in .NET 7 code - where the transaction has been promoted to a distributed transaction and work has taken place in another process - the code blocks indefinitely.
Regression?
This code works correctly in .NET Framework 4.8
Known Workarounds
No response
Configuration
.NET 7 RC1
Windows Server 2019 Standard - 1809 (17763.2565)
x64
Other information
No response
The text was updated successfully, but these errors were encountered: