Feature: Support for DSN flag - sendStringParametersAsUnicode

[sinhashubham95]

This is a feature addition for a new DSN flag - sendStringParametersAsUnicode which basically allows us to specify if the Go default string types should be sent as UNICODE(if true) or not(if false).

Official Support for sendStringParametersAsUnicode in JDBC

[sinhashubham95]

@microsoft-github-policy-service agree

[sinhashubham95]

@stuartpa @apoorvdeshmukh @shueybubbles can you please check this PR?

[shueybubbles]

ing types sent as varchar.

without any way of indicating the code page, how does this work? Go strings are UTF8

[sinhashubham95]

@shueybubbles it's upto the clients to decide how they want to use Go strings. In the same way other drives like postgres for Go also works.

[shueybubbles]

I'm not convinced it's a good idea for the driver to have such a broad toggle without a more comprehensive fix for mixing string encodings AND having a way for clients to specify the code page of the input.

Note the driver already has a way to designate input parameter strings as varchar to avoid the conversion - you need to use the driver-specific VarChar
Using that type has the benefit of apps being explicit about the desired conversion behavior and being able to mix unicode strings with non-unicode strings in the same connection.

[sinhashubham95]

@shueybubbles the fact that this flag is made available in the official support for sqlserver is because it makes the applications which work by default with non-unicode strings have lesser chances of failure. With this once and for all we can add this in the connection string, and nowhere explicit type casts are required on top of a string to varchar, without which any database columns having varchar as a type, will work the way as expected(for example index queries, etc). We are just trying to adhere to the official documentation of sqlserver here.

[sinhashubham95]

@shueybubbles can you help close this? This has been pending since long. We can have a quick connect on this if needed.

[shueybubbles]

please add some functional tests showing what problem this is solving.
JDBC differentiates parameters by type and only applies this flag when the input parameter is not a wide character type. Your change would apply the conversion to all input strings regardless of whether the server type is a wide char type which seems incorrect.

[moizm89]

@shueybubbles problem this PR solves is mentioned here - #40

[sinhashubham95]

@shueybubbles I agree to your point that it will change for all input parameters that are of the string type, but it will only happen for the clients opting for it specifically. It's a backward compatible change as by default this flag is true itself unless specified.

[sinhashubham95]

@shueybubbles have added test cases for the above please check, also this feature is in accordance with the Official JDBC driver as well.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 85.71429% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 74.54%. Comparing base (fe7c3d4) to head (28f3fbf).
Report is 2 commits behind head on main.

❗ Current head 28f3fbf differs from pull request most recent head 0b266b8. Consider uploading reports for the commit 0b266b8 to get more accurate results

Files	Patch %	Lines
mssql.go	85.71%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #180      +/-   ##
==========================================
- Coverage   74.72%   74.54%   -0.18%     
==========================================
  Files          32       32              
  Lines        6298     6341      +43     
==========================================
+ Hits         4706     4727      +21     
- Misses       1316     1331      +15     
- Partials      276      283       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shueybubbles]

@shueybubbles have added test cases for the above please check, also this feature is in accordance with the Official JDBC driver as well.

Your proposed change is quite different from the JDBC model.
In JDBC, the application can still use alternative methods and types to pass UNICODE data to the server. With your proposal, there's no way for an app to say "please send this string as unicode" .
By comparison, with the current VarChar type declared in the go-mssqldb driver, it's entirely possible for an app to declare a parameter as MBCS (really, UTF-8) so it doesn't get converted to UTF-16 on the way to the server. What's the default MBCS encoding for a JDBC string? Is it system-dependent? If so, comparing that model to the Go model where strings are always UTF8 is dicy.

As the docs for JDBC point out, blindly sending non-unicode data to the server creates a lot of chances for encoding errors due to collation mismatches. If someone just changes the connection string without changing their code, apps that rely on the current behavior to avoid those mismatches will be broken, in ways that can be hard to detect until much later when someone notices data quality issues.