Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SqlTypes members to remove SqlClient internal access hacks #51836

Closed
Wraith2 opened this issue Apr 25, 2021 · 40 comments · Fixed by #72724
Closed

Add SqlTypes members to remove SqlClient internal access hacks #51836

Wraith2 opened this issue Apr 25, 2021 · 40 comments · Fixed by #72724
Assignees
Labels
api-approved API was approved in API review, it can be implemented area-System.Data.SqlClient
Milestone

Comments

@Wraith2
Copy link
Contributor

Wraith2 commented Apr 25, 2021

Background and Motivation

In the desktop framework the System.Data.SqlClient namespace was originally part of the System.Data assembly allowing SqlClient to access internal methods of some types in the System.Data.SqlTypes namespace. In netcoreSqlClient was split out into a separate assembly but rather than add new public apis to the SqlTypes struct overlay hacks were used to maintain the existing access to internals.

SqlClient has now been split out of the framework entirely into a separate project Microsoft.Data.SqlClient and that project still uses the overlay hack to access internals, all collected into the [SqlTypeWorkarounds](SqlClient/SqlTypeWorkarounds.cs at main · dotnet/SqlClient (github.com)) class. It is unlikely that the layout or definition of the SqlTypes members will change but it is undesirable to have a first party library use such access when it is possible and relatively simple to extend the public api.

Proposed API

namespace System.Data.SqlTypes
{
	public struct SqlDateTime
	{
+   	public static DateTime FromInternalRepresentation(int daypart, int timepart)
	}
	
	public struct SqlMoney
    {
+		public static SqlMoney FromInternalRepresentation(long value);
+		public long GetInternalRepresentation();
    }
    
    public struct SqlDecimal
    {
+		public GetInternalDataRepresentation(Span<uint> value)
    }
    
    public struct SqlBinary
    {
+		public static SqlBinary WrapBytes(byte[] bytes)    
    }
    
    public struct SqlGuid
    {
+		public static SqlGuid WrapBytes(byte[] bytes)
    }
}

Usage Examples

Creating an SqlMoney from it's internal TDS representation will change from:

...
	case TdsEnums.SQLUNIQUEID:
		value.SqlGuid = SqlTypeWorkarounds.SqlGuidCtor(unencryptedBytes, true);   // doesn't copy the byte array
        break;
...
    
public static class SqlTypeWorkarounds
{
    internal static SqlGuid SqlGuidCtor(byte[] value, bool ignored)
    {
        var c = default(SqlGuidCaster);
        c.Fake._value = value;
        return c.Real;
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct SqlGuidLookalike
    {
        internal byte[] _value;
    }

    [StructLayout(LayoutKind.Explicit)]
    private struct SqlGuidCaster
    {
        [FieldOffset(0)]
        internal SqlGuid Real;
        [FieldOffset(0)]
        internal SqlGuidLookalike Fake;
    }
}

to:

...
	case TdsEnums.SQLUNIQUEID:
		value.SqlGuid = SqlGuid.WrapBytes(unencryptedBytes);
        break;
...

Alternative Designs/Reasoning

SqlDateTime.FromInternalRepresentation

  • SqlDateTime has an internal static method internal static DateTime ToDateTime(int daypart, int timepart) which could have the access modifier changed to public. I chose not to suggest this because it doesn't convert an SqlDateTime instance to a DateTime, it converts parameters which are the internal representation used in TDS into a DateTime, the input and output are not what a user would expect without reading the documentation.
  • It could be argued that the return value should be an SqlDateTime however this is not how it is used and if an SqlDateTime were used it would usually need to be unwrapped because a user requested a known-not-null DateTime through GetFieldValue<DateTime> or GetDateTime

SqlMoney.ToSqlInternalRepresentation

SqlMoney already contains an internal implementation of this method so i propose only to change the existing access modifier. The internal representation is an unscaled value read from a TDS packet.

SqlMoney.FromInternalRepresentation

SqlMoney contains an internal ctor internal SqlMoney(long value, int ignored) which was used to perform an unscaled construction of SqlMoney from the internal representation. The ignored parameter is present because there is already a public ctor internal SqlMoney(long value) which performs a scaled creation. Since I've already proposed a method to get the unscaled value and contains the words InternalRepresentation it made sense to me to have a symmetric static method to do the creation from that representation.

SqlDecimal.GetInternalDataRepresentation

This method name was chosen for close to be similarity to the SqlMoney and SqlDateTime methods which provide access to internal data representations. This method does not give access to all of the internal data for the SqlDecimal, it only provides access to 4 Data uints which contain the value, it does not provide access to the additional 4 bytes of data (Status, Length, Precision and Scale) and because of that I have added the word Data in the middle of the name. Access to the other byte fields is done through existing public accessor properties.

This method could have been called GetBits for some symmetry with decimal.GetBits, i decided against this because it isn't an exactly analogous operation.

SqlBinary.WrapBytes and SqlGuid.WrapBytes

These could be called TakeBytes but that felt clumsy or too low level a name. Their intention is to return an instance of their containing type which uses the reference to the array parameter passed in, currently byte[] ctor for each type would allocate a new appropriately sized array and copy the contents of the parameter into it.

General

The existing behaviour of accessing internal members will be present in pre NET6 framework and SqlClient binaries for as long as both exist. It would cause compatibility problems discoverable only at runtime if we tried to change that. Adding these members will allow the same access to be done using a legitimate channel on NET6 and future SqlClient versions that compile against it.

Adding these new members simply makes the access to data used by SqlClient official and thus documented. They should have no performance impact.

Risks

The new members will need to be documented.

/cc area owners @ajcvickers, @roji and @David-Engel

@Wraith2 Wraith2 added the api-suggestion Early API idea and discussion, it is NOT ready for implementation label Apr 25, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Data.SqlClient untriaged New issue has not been triaged by the area owner labels Apr 25, 2021
@ghost
Copy link

ghost commented Apr 25, 2021

Tagging subscribers to this area: @cheenamalhotra, @David-Engel
See info in area-owners.md if you want to be subscribed.

Issue Details

Background and Motivation

In the desktop framework the System.Data.SqlClient namespace was originally part of the System.Data assembly allowing SqlClient to access internal methods of some types in the System.Data.SqlTypes namespace. In netcoreSqlClient was split out into a separate assembly but rather than add new public apis to the SqlTypes struct overlay hacks were used to maintain the existing access to internals.

SqlClient has now been split out of the framework entirely into a separate project Microsoft.Data.SqlClient and that project still uses the overlay hack to access internals, all collected into the [SqlTypeWorkarounds](SqlClient/SqlTypeWorkarounds.cs at main · dotnet/SqlClient (github.com)) class. It is unlikely that the layout or definition of the SqlTypes members will change but it is undesirable to have a first party library use such access when it is possible and relatively simple to extend the public api.

Proposed API

namespace System.Data.SqlTypes
{
	public struct SqlDateTime
	{
+   	public static DateTime FromInternalRepresentation(int daypart, int timepart)
	}
	
	public struct SqlMoney
    {
+		public static SqlMoney FromInternalRepresentation(long value);
+		public long GetInternalRepresentation();
    }
    
    public struct SqlDecimal
    {
+		public GetInternalDataRepresentation(Span<uint> value)
    }
    
    public struct SqlBinary
    {
+		public static SqlBinary WrapBytes(byte[] bytes)    
    }
    
    public struct SqlGuid
    {
+		public static SqlBinary WrapBytes(byte[] bytes)
    }
}

Usage Examples

Creating an SqlMoney from it's internal TDS representation will change from:

...
	case TdsEnums.SQLUNIQUEID:
		value.SqlGuid = SqlTypeWorkarounds.SqlGuidCtor(unencryptedBytes, true);   // doesn't copy the byte array
        break;
...
    
public static class SqlTypeWorkarounds
{
    internal static SqlGuid SqlGuidCtor(byte[] value, bool ignored)
    {
        var c = default(SqlGuidCaster);
        c.Fake._value = value;
        return c.Real;
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct SqlGuidLookalike
    {
        internal byte[] _value;
    }

    [StructLayout(LayoutKind.Explicit)]
    private struct SqlGuidCaster
    {
        [FieldOffset(0)]
        internal SqlGuid Real;
        [FieldOffset(0)]
        internal SqlGuidLookalike Fake;
    }
}

to:

...
	case TdsEnums.SQLUNIQUEID:
		value.SqlGuid = SqlGuid.WrapBytes(unencryptedBytes);
        break;
...

Alternative Designs/Reasoning

SqlDateTime.FromInternalRepresentation

  • SqlDateTime has an internal static method internal static DateTime ToDateTime(int daypart, int timepart) which could have the access modifier changed to public. I chose not to suggest this because it doesn't convert an SqlDateTime instance to a DateTime, it converts parameters which are the internal representation used in TDS into a DateTime, the input and output are not what a user would expect without reading the documentation.
  • It could be argued that the return value should be an SqlDateTime however this is not how it is used and if an SqlDateTime were used it would usually need to be unwrapped because a user requested a known-not-null DateTime through GetFieldValue<DateTime> or GetDateTime

SqlMoney.ToSqlInternalRepresentation

SqlMoney already contains an internal implementation of this method so i propose only to change the existing access modifier. The internal representation is an unscaled value read from a TDS packet.

SqlMoney.FromInternalRepresentation

SqlMoney contains an internal ctor internal SqlMoney(long value, int ignored) which was used to perform an unscaled construction of SqlMoney from the internal representation. The ignored parameter is present because there is already a public ctor internal SqlMoney(long value) which performs a scaled creation. Since I've already proposed a method to get the unscaled value and contains the words InternalRepresentation it made sense to me to have a symmetric static method to do the creation from that representation.

SqlDecimal.GetInternalDataRepresentation

This method name was chosen for close to be similarity to the SqlMoney and SqlDateTime methods which provide access to internal data representations. This method does not give access to all of the internal data for the SqlDecimal, it only provides access to 4 Data uints which contain the value, it does not provide access to the additional 4 bytes of data (Status, Length, Precision and Scale) and because of that I have added the word Data in the middle of the name. Access to the other byte fields is done through existing public accessor properties.

This method could have been called GetBits for some symmetry with decimal.GetBits, i decided against this because it isn't an exactly analogous operation.

SqlBinary.WrapBytes and SqlGuid.WrapBytes

These could be called TakeBytes but that felt clumsy or too low level a name. Their intention is to return an instance of their containing type which uses the reference to the array parameter passed in, currently byte[] ctor for each type would allocate a new appropriately sized array and copy the contents of the parameter into it.

General

The existing behaviour of accessing internal members will be present in pre NET6 framework and SqlClient binaries for as long as both exist. It would cause compatibility problems discoverable only at runtime if we tried to change that. Adding these members will allow the same access to be done using a legitimate channel on NET6 and future SqlClient versions that compile against it.

Adding these new members simply makes the access to data used by SqlClient official and thus documented. They should have no performance impact.

Risks

The new members will need to be documented.

/cc area owners @ajcvickers, @roji and @David-Engel

Author: Wraith2
Assignees: -
Labels:

api-suggestion, area-System.Data.SqlClient, untriaged

Milestone: -

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jun 7, 2021

@roji , @David-Engel triage?

@David-Engel
Copy link
Contributor

This makes sense to me. Seems like one of the easier internals hacks to resolve in SqlClient since the breakup.

@roji @ajcvickers @saurabh500 @cheenamalhotra - Anyone object to exposing the proposed methods in System.Data.SqlTypes as public APIs?

@cheenamalhotra
Copy link
Member

cheenamalhotra commented Jun 7, 2021

My initial thoughts:

  1. Can we name APIs: SqlBinary.WrapBytes and SqlGuid.WrapBytes to:
    "SqlBinary.FromByteArray(byte [])" and "SqlGuid.FromByteArray(byte [])"
    This would keep the nomenclature similar to existing Types where an API instantiates an instance from another type, e.g. DateTime.FromBinary? Also "Wrap/Unwrap" terms are used for "Encrypt/Decrypt" interchangeably.
  2. I don't see any .NET API with "internaldatarepresentation" naming convention. Could we rename these get and from APIs to their specific types, e.g. ToLong()/ FromLong(long) for SqlMoney and for SqlDateTime construction, you could call it FromDateTimeParts?
  3. What is the return type of SqlDecimal.GetInternalDataRepresentation ? It's name could also be derived based on what it returns as ToXxxx().

My thought is what is an internal data representation doesn't need to be called out specifically in API name. Conversion can be handled "From" and "To" a datatype just like every other API in base Types, unless the type itself does not represent the data and needs a specific definition.

@roji
Copy link
Member

roji commented Jun 8, 2021

  • I'm assuming these changes are being proposed for .NET Core; if so, wouldn't the hacky workarounds have to continue being there for the .NET Framework version of Microsoft.Data.SqlClient? If so, is there value in doing this (and further driving the two codebases apart)?
  • Does a similar problem exist on the writing side, i.e. does SqlClient need a ToInternalDataRepresentation as well (or are these already exposed in some other ways)?
  • For SqlBinary and SqlGuid, I'd lean towards publicly exposing the 2nd constructor with a createCopy flag, rather than a static WrapBytes/FromByteArray - mainly because you already have a constructor accepting byte[] (so it would be more consistent/discoverable etc.). But that's a minor API shape question.
  • I'll look at the other APIs more closely a bit later.

For the record, I'd have much preferred for these types to be copied into Microsoft.Data.SqlClient - but that does seem impractical.

@roji
Copy link
Member

roji commented Jun 8, 2021

I'm assuming these changes are being proposed for .NET Core; if so, wouldn't the hacky workarounds have to continue being there for the .NET Framework version of Microsoft.Data.SqlClient? If so, is there value in doing this (and further driving the two codebases apart)?

BTW, unless I'm mistaken the same holds for previous .NET Core versions - the earlier point when SqlTypeWorkarounds would be removable is when M.D.SqlClient drops support for net5.0, right? Is that worth pursuing?

@Joe4evr
Copy link
Contributor

Joe4evr commented Jun 8, 2021

  1. Can we name APIs: SqlBinary.WrapBytes and SqlGuid.WrapBytes to:
    "SqlBinary.FromByteArray(byte [])" and "SqlGuid.FromByteArray(byte [])"

Admittedly, I don't know much about these SQL types, but I don't understand the request for these factory methods, given that both of those already have a constructor that takes in a byte[].

@roji
Copy link
Member

roji commented Jun 8, 2021

@Joe4evr if you look at the implementation, you'll see these constructor copy the given byte array, which isn't good for perf and isn't needed when SqlClient is constructing the instance.

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jun 8, 2021

Can we name APIs: SqlBinary.WrapBytes and SqlGuid.WrapBytes to:
"SqlBinary.FromByteArray(byte [])" and "SqlGuid.FromByteArray(byte [])"

I considered this but decided to use Wrap and Unwrap because it clearly communicates that the array passed in it taken and not copied which is important for correct use.

I don't see any .NET API with "internaldatarepresentation" naming convention. Could we rename these get and from APIs to their specific types, e.g. ToLong()/ FromLong(long) for SqlMoney and for SqlDateTime construction, you could call it FromDateTimeParts?

The values returned being internal representations is important. The value contained in the returned data type is not the usual format that .net programs expect and should not be treated as convertible by standard apis.

What is the return type of SqlDecimal.GetInternalDataRepresentation ? It's name could also be derived based on what it returns as ToXxxx().

void is the return type, the span argument is filled with the bytes. It could be made bool or OperationStatus if that is more appropriate. Failure to pass in a correctly sized buffer is an exception state imo.

I'm assuming these changes are being proposed for .NET Core; if so, wouldn't the hacky workarounds have to continue being there for the .NET Framework version of Microsoft.Data.SqlClient? If so, is there value in doing this (and further driving the two codebases apart)?

Yes, netcore only because netfx is frozen. There are always going to have to be separate code paths for the two frameworks so this doesn't add much extra burden. It will allow the type workarounds to be clearly marked as deprecated in tfms that use them, they should not be added to. It is not going to be possible to remove the existing hacks for the forseeable future.

Being able to remove them and being able to say that they should not be used or extended anymore are two different goals. As an external library to the runtime SqlClient should not be using private implementation detail information like this, when it was done it was for pragmatic reasons but leaving it like this can be seen as implicit blessing and it should not be.

Does a similar problem exist on the writing side, i.e. does SqlClient need a ToInternalDataRepresentation as well (or are these already exposed in some other ways)?

An existing operation handles the to direction conversion.

I'd have much preferred for these types to be copied into Microsoft.Data.SqlClient

I agree but without that wouldn't be possible without breaking DataTable and DataAdapter interoperability. Those are still supported classes and likely to be in use a huge amount of LOB codebases.

@roji
Copy link
Member

roji commented Jun 8, 2021

Yes, netcore only because netfx is frozen. There are always going to have to be separate code paths for the two frameworks so this doesn't add much extra burden. It will allow the type workarounds to be clearly marked as deprecated in tfms that use them, they should not be added to. It is not going to be possible to remove the existing hacks for the forseeable future.

Not just netfx is frozen - also older versions of netcore. You'll need to multi-target to make use of this.

Being able to remove them and being able to say that they should not be used or extended anymore are two different goals. As an external library to the runtime SqlClient should not be using private implementation detail information like this, when it was done it was for pragmatic reasons but leaving it like this can be seen as implicit blessing and it should not be.

That's certainly true in theory, but once again, this is going to stay in place until M.D.SqlClient drops net5.0 (which I think is, well, an extremely long time). And while SqlTypes is technically part of the runtime, but in practice is part of SqlClient (we just can't copy those types out because of DataTable). So speaking very pragmatically, I think there's very little value here, and personally wouldn't spend any time on it (and the API design discussions it could spawn e.g. regarding naming).

However, at the end of the day, since I view SqlTypes as internal to SqlClient, it's ultimately up to you guys. If you really think this is worth pursuing we can try.

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jun 9, 2021

SqlClient already multi targets and probably always will. As you say, the future where net6 is the lowest supported runtime is a long time away and may never occur.

I think it is worth doing because:

  1. it removes rule breaking of SqlClient that can be cited as "Well MS do it themselves so I can too". We should not break rules that we want others to follow.
  2. It makes the interface between SqlClient and System.Data.SqlTypes explicit so in future if they are removable it's clear what is being used.
  3. It's relatively low effort

If someone feels that any of those points are incorrect then we can just carry on as we do currently.

@Wraith2
Copy link
Contributor Author

Wraith2 commented Oct 28, 2021

What is this waiting for please?

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jan 13, 2022

Any chance of an update? This seems to be in limbo as untriaged. Is it not at least worth letting the design team look at it and decide whether it has any merit?

@roji
Copy link
Member

roji commented Jan 13, 2022

Just to be clear, this would be waiting on the SqlClient team, should they decide they want to do this.

I personally still believe this has very little value:

  1. This is very old code - it's been this way forever, and code around this is extremely unlikely to change.
  2. It's very unlikely that these types or the APIs in question will be removed in the future.
  3. Any churn in the runtime carries some risk
  4. There's very little value in actually doing this

But as I wrote above, this is ultimately SqlClient's decision.

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jan 13, 2022

/cc @David-Engel @DavoudEshtehari

@David-Engel
Copy link
Contributor

While I see that this is the "right" thing to do, I'll be honest. We don't have the bandwidth to push for it. If you (@Wraith2) want to submit the API and PR on the runtime side and the equivalent changes on the SqlClient side, I'm not against that. But you'd be waiting for us to create a suitable .NET x specific target and this feature alone isn't strong enough to warrant its own target. We will likely be creating a .NET 6 target (skipping 5 since it's out of support in May) and skipping 7 since I don't think it will be
LTS.

I think you might get frustrated when people aren't prioritizing each required change.

There are many things I would prioritize over this. Examples:

  1. I would love to see this addressed --> Remove dependency on .NET Core internal shims SqlClient#303
  2. Difficult ones, I know: Queries with MultipleActiveResultSets=True (MARS) are very slow / time out on Linux SqlClient#422 and Execution Timeout Expired Error (258, ReadSniSyncOverAsync) SqlClient#647
  3. .NET 6 | Support the new BCL DateOnly and TimeOnly structs SqlClient#1009
  4. .NET 5.0 | Implement SqlException.IsTransient SqlClient#649
  5. .NET 5.0 | Implement SqlException.SqlState SqlClient#648
  6. .NET 5.0 | Add async System.Data resultset and database schema APIs  SqlClient#646
  7. .NET 5.0 | Implement the ADO.NET transaction savepoint APIs SqlClient#1256
  8. Implement async transaction commit/rollback methods SqlClient#113

Regards,
David

@Wraith2
Copy link
Contributor Author

Wraith2 commented Jan 14, 2022

Ok.

@David-Engel
Copy link
Contributor

Re-opening issue for consideration in a future .NET release.

@David-Engel David-Engel reopened this Jun 17, 2022
@roji roji added this to the Future milestone Jun 18, 2022
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Jun 18, 2022
@jeffhandley jeffhandley added the api-ready-for-review API is ready for review, it is NOT ready for implementation label Jul 7, 2022
@jeffhandley jeffhandley modified the milestones: Future, 7.0.0 Jul 7, 2022
@jeffhandley
Copy link
Member

Thanks, @Wraith2. I updated this specific issue and fired off an email to resolve the permissions snag.

@terrajobst
Copy link
Member

terrajobst commented Jul 19, 2022

Video

  • SqlDateTime
    • The name FromInternalRepresentation feels off
    • Why does SqlDateTime.FromInternalRepresentation return a DateTime and not a SqlDateTime?
    • Do we need this at all?
    • How about public static SqlDateTime FromParts(int dayPart, int timePart)
  • SqlMoney
    • How about FromTdsValues to ToTdsValue? This makes it clear what "internal representation" it refers to.
  • SqlDecimal
    • Should return the number of elements written to the supplied span
    • int GetTdsValue(Span<uint> destination)
    • Don't you need a SqlDecimal FromTdsValue(ReadOnlySpan<uint> source)?
    • Why int? The constructor takes them as four int values.
  • SqlBinary, SqlGuid
    • APIs that don't copy are unusual, but this works
namespace System.Data.SqlTypes;

public partial struct SqlDateTime
{
    public static SqlDateTime FromParts(int dayPart, int timePart);
}

public partial struct SqlMoney
{
    public static SqlMoney FromTdsValue(long value);
    public long GetTdsValue();
}

public partial struct SqlDecimal
{
    public int WriteTdsValue(Span<uint> destination);
}

public partial struct SqlBinary
{
    public static SqlBinary WrapBytes(byte[] bytes);
}

public partial struct SqlGuid
{
    public static SqlGuid WrapBytes(byte[] bytes);
}

@terrajobst terrajobst added api-approved API was approved in API review, it can be implemented and removed api-ready-for-review API is ready for review, it is NOT ready for implementation labels Jul 19, 2022
@Wraith2
Copy link
Contributor Author

Wraith2 commented Jul 19, 2022

I posted in Youtube chat during the api review but it doesn't look like people can see all of the chat messages.
Is this approved for 7 or 8?

@bartonjs
Copy link
Member

@Wraith2 Generally the API Review meeting doesn't talk about releases (other than "this looks scary big and you probably won't get feedback before it's frozen, you should probably wait for the next release"). The issue is marked as 7, so I assume the intent was to get it in for .NET 7.

@GrabYourPitchforks
Copy link
Member

I missed review this morning due to a conflict but have a concern. I believe we should not create SqlGuid.WrapBytes. The other APIs in the proposal are ok.

If we create SqlGuid.WrapBytes, the contracted behavior would be that the SqlGuid instance wraps the byte array, meaning that any changes to the byte array must be reflected back to the new SqlGuid instance. This implies that the layout of SqlGuid is forever locked to the following:

public struct SqlGuid
{
    private byte[] _value;
}

And we would never be able to change the internal layout to this more optimized shape:

public struct SqlGuid
{
    private Guid? _value;
}

This hampers our ability to make future improvements to the Framework. If we are really concerned about the performance of the SqlGuid type, we should just make perf improvements directly to its implementation without introducing any new API.

@terrajobst
Copy link
Member

If we create SqlGuid.WrapBytes, the contracted behavior would be that the SqlGuid instance wraps the byte array, meaning that any changes to the byte array must be reflected back to the new SqlGuid instance.

Yes, but. The type is in the framework since .NET 1.0. It's not to say that we can't improve the type, it just seems unlikely at this point...

@GrabYourPitchforks
Copy link
Member

GrabYourPitchforks commented Jul 19, 2022

It's not to say that we can't improve the type, it just seems unlikely at this point...

Yet here we are, in .NET 7, talking about improving the type. And API review gave a stamp of approval to doing so. :) So I don't buy that it's "unlikely".

This issue was opened literally because there's a performance problem with the existing implementation. Why aren't we doing the easy non-API-changing thing to try fixing the implementation first, before considering new API which permanently prevents us from fixing the implementation?

@terrajobst
Copy link
Member

terrajobst commented Jul 19, 2022

Yet here we are, in .NET 7, talking about improving the type.

LOL. I was more thinking "the fundamental shape or encoding of the type". Adding methods that operate over the same shape is much easier to reason about.

This issue was opened literally because there's a performance problem with the existing implementation. Why aren't we doing the easy non-API-changing thing to try fixing the implementation first, before considering new API which permanently prevents us from fixing the implementation?

That's a great question and @David-Engel would be more qualified to answer that. I assume these types are kinda frozen in their shape due to being tied to the TDS spec.

@GrabYourPitchforks
Copy link
Member

I assume these types are kinda frozen in their shape due to being tied to the TDS spec.

The TDS spec freezes the capabilities of these types, but it doesn't freeze their layout details. We had quite the in-depth email discussion on this a few months ago.

@David-Engel
Copy link
Contributor

This issue was opened literally because there's a performance problem with the existing implementation. Why aren't we doing the easy non-API-changing thing to try fixing the implementation first, before considering new API which permanently prevents us from fixing the implementation?

That's a great question and @David-Engel would be more qualified to answer that. I assume these types are kinda frozen in their shape due to being tied to the TDS spec.

To clarify, I interpret the "fix the implementation" statement to mean fix the System.Data.SqlTypes implementation of new SqlGuid(byte[] value) where it copies the byte array. (SqlBinary does the same thing.)

SqlClient simply needs methods that avoid so much unnecessary byte copying, as the whole job of the database client is to construct many, many objects like these. I'm fine with whatever API shape we land on to accomplish that.

@bartonjs
Copy link
Member

I think @GrabYourPitchforks 's version of "fix the implementation" is to change the field from byte[] to Guid (or Guid?), and invert the relationship of how the byte[] ctor and Guid ctor work. Then the impl is storing values in the GUID-sized struct, not storing an array at all.

@MichalPetryka
Copy link
Contributor

Since the struct seems to keep a separate state of null, I'd say that Guid? would be more applicable here.

@GrabYourPitchforks
Copy link
Member

GrabYourPitchforks commented Jul 20, 2022

Right. "Fixing the implementation" wouldn't require adding any new API shape at all. It would literally be changing this:

private byte[] _value;
public SqlGuid(byte[] value)
{
    // argument length check
    _value = (byte[])value.Clone();
}

To:

private Guid? _value;
public SqlGuid(byte[] value)
{
    // argument length check
    _value = new Guid(value);
}

Aside from the length check (which you'd need in all cases), it's literally a single movdqu instruction and a single xor instruction on a 32-bit register. You can't really get much faster than that. (In fact, this has the added benefit that it doesn't need to update the GC card table since you're only storing references to value types, not references to reference types!)

@MichalPetryka
Copy link
Contributor

FYI: optimizing the implementation would require implementing ISerializable so that binary serialization keeps working.

MichalPetryka added a commit to MichalPetryka/runtime that referenced this issue Jul 20, 2022
Optimizes the SqlGuid struct according to the idea
from dotnet#51836.
@MichalPetryka
Copy link
Contributor

MichalPetryka commented Jul 20, 2022

I've opened a draft to show how the proposed implementation changes would look like: #72549

@terrajobst
Copy link
Member

@David-Engel, what your thoughts on this? What @GrabYourPitchforks said makes sense to me.

@David-Engel
Copy link
Contributor

David-Engel commented Jul 20, 2022

@David-Engel David Engel (Simba Technologies Inc) Vendor, what your thoughts on this? What @GrabYourPitchforks Levi Broderick FTE said makes sense to me.

Agreed. I'm all for PR #72549. We'll have to adjust/service System.Data.SqlClient and Microsoft.Data.SqlClient to keep them compatible with .NET 7. With #72549, the API proposal here can eliminate the new public static SqlGuid WrapBytes(byte[] bytes); method.

@MichalPetryka
Copy link
Contributor

MichalPetryka commented Jul 20, 2022

Instead this is needed in order to maintain binary serialization (is a new API review needed?):

public partial struct SqlGuid : ISerializable
{
    void ISerializable.GetObjectData(SerializationInfo si, StreamingContext sc);
}

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 23, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 27, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Aug 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
api-approved API was approved in API review, it can be implemented area-System.Data.SqlClient
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants