Span compat #757

martijn00 · 2023-03-23T13:24:35Z

Add netstandard support for Span on: #754

ebozduman · 2023-03-30T00:13:16Z

First, I see the following issue when I run functional tests against 4.8:

/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj : error NU1605: Warning As Error: Detected package downgrade: System.ValueTuple from 4.5.0 to 4.4.0. Reference the package directly from the project to select a different version. 
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj : error NU1605:  Minio.Functional.Tests -> Minio -> System.Text.Json 7.0.2 -> System.ValueTuple (>= 4.5.0) 
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj : error NU1605:  Minio.Functional.Tests -> Minio -> System.ValueTuple (>= 4.4.0)

After fixing it by simply bumping up the version of System.ValueTuple package to 4.5.0, I see the following:

$ dotnet run --project ./Minio.Functional.Tests/Minio.Functional.Tests.csproj
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/RandomStreamGenerator.cs(28,27): error CS1503: Argument 1: cannot convert from 'System.Span<byte>' to 'byte[]' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(720,62): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(721,42): error CS1061: 'FileStream' does not contain a definition for 'DisposeAsync' and no accessible extension method 'DisposeAsync' accepting a first argument of type 'FileStream' could be found (are you missing a using directive or an assembly reference?) [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(804,62): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(805,42): error CS1061: 'FileStream' does not contain a definition for 'DisposeAsync' and no accessible extension method 'DisposeAsync' accepting a first argument of type 'FileStream' could be found (are you missing a using directive or an assembly reference?) [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(885,62): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(886,42): error CS1061: 'FileStream' does not contain a definition for 'DisposeAsync' and no accessible extension method 'DisposeAsync' accepting a first argument of type 'FileStream' could be found (are you missing a using directive or an assembly reference?) [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(926,33): error CS0117: 'File' does not contain a definition for 'ReadAllBytesAsync' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(960,33): error CS0117: 'File' does not contain a definition for 'ReadAllBytesAsync' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(1498,41): error CS0117: 'MD5' does not contain a definition for 'HashData' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(1500,38): error CS0117: 'MD5' does not contain a definition for 'HashData' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(2612,41): error CS1503: Argument 1: cannot convert from 'string' to 'char' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(2697,49): error CS0103: The name 'HttpUtility' does not exist in the current context [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4574,62): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4575,42): error CS1061: 'FileStream' does not contain a definition for 'DisposeAsync' and no accessible extension method 'DisposeAsync' accepting a first argument of type 'FileStream' could be found (are you missing a using directive or an assembly reference?) [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4717,28): error CS0117: 'File' does not contain a definition for 'WriteAllLinesAsync' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4750,66): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4751,46): error CS1061: 'FileStream' does not contain a definition for 'DisposeAsync' and no accessible extension method 'DisposeAsync' accepting a first argument of type 'FileStream' could be found (are you missing a using directive or an assembly reference?) [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4758,61): error CS0117: 'File' does not contain a definition for 'ReadAllTextAsync' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4810,35): error CS0117: 'File' does not contain a definition for 'OpenHandle' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4811,57): error CS1503: Argument 2: cannot convert from 'System.IO.FileAccess' to 'System.IO.FileMode' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]
/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/FunctionalTest.cs(4827,48): error CS1503: Argument 2: cannot convert from 'System.Threading.CancellationToken' to 'int' [/home/ersan/work/src/github.com/minio/minio-dotnet/Minio.Functional.Tests/Minio.Functional.Tests.csproj]

The build failed. Fix the build errors and run again.

I started looking into these, but then I had to switch to another task. If you'd like I can send you the diffs of the files I've touched.

martijn00 · 2023-03-30T19:00:51Z

@ebozduman Why would you switch that to 4.8? It is even on 6.0 on the older branches: https://github.com/minio/minio-dotnet/blob/4.0.7/Minio.Functional.Tests/Minio.Functional.Tests.csproj#L3

It never worked on net4.8. The library will be compatible with 4.8 this way, but the test are just an output, and the code itself should not be compatible.

martijn00 · 2023-03-30T19:28:02Z

@ebozduman Anyways, i've added the support for net48 in the tests. Can you get the nuget release out please?

ebozduman · 2023-03-30T19:57:26Z

@martijn00,
.NET4.x was supposed to be tested and supported since from the first day. So, we need to fix any incompatibility issues, add required testing in CI process and fully support it.

Thank you for working on this task.

martijn00 · 2023-03-30T20:20:50Z

@ebozduman I've just fixed the failing test

ebozduman · 2023-04-03T09:43:23Z

@martijn00

I will push some changes in a short while. These changes go on top of your PR, #757, and it completes the changes required to support older 4.x dotnet releases together with 6.0 and 7.0 dotnet releases we support. I ran all functional tests against all the supported releases, 4.7.2, 4.8, 4.8.1, 6.0, 7.0, and they all passed successfully.

However, I expect the dotnet regitlint check to fail during the CI build process because of a bug in JetBrains.ReSharper.Psi.ExtensionsAPI.Tree.ModificationUtil.DeleteChildRange.
I am looking into it and I think there is a workaround to it.

I appreciate if you could review my changes in the meantime.

As soon as the lint check issue is resolved and the review is complete, we can merge PR#757 and start a new release.

ebozduman · 2023-04-03T14:16:00Z

@martijn00 ,

Please disregard the last 2 commits. It is my bad (I was sleep deprived). I'll fix it.

martijn00 · 2023-04-03T19:45:17Z

@ebozduman hehe alright, I'll wait a bit then because you did some strange things in there. Also I want to say again, lets get a release out first and then follow up with more fixes and warning updates because this is getting way too big.

ebozduman · 2023-04-03T21:12:35Z

Yes @martijn00 ,
I agree.
I had to reverse some of your changes to make the code work for 4.x and for core releases.
We can make the code better as soon as a new Minio package is built and released.

martijn00 · 2023-04-04T08:31:45Z

@ebozduman what are you doing with those commits? Please revert because it is not correct. Let me know what you want so I can have a look if changes need to be made.

ebozduman · 2023-04-04T22:11:01Z

You're right. I'll clean it up.
Sorry for the mess.

ebozduman · 2023-04-05T02:18:28Z

Done. My recent commit is removed.

As soon as, the build testing is a PASS, we can merge the PR.

Could you please look into the build lint failures?
When the lint issues are addressed, there will be 2 more little fixes we'll need:

1- Func. test ObjectLockConfigurationAsync_Test1 fails

System.InvalidOperationException: There was an error reflecting type
'Minio.DataModel.Tags.Tagging'.
 ---> System.NotSupportedException: Cannot serialize member
Minio.DataModel.Tags.Tagging.Tags of type
System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib,
Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.String,
System.Private.CoreLib, Version=6.0.0.0, Culture=neutral,
PublicKeyToken=7cec85d7bea7798e]], because it implements
 IDictionary.
:
:

2- Mint.Logger.Log() function escapes some chars, special chars if any appears in the object name, function or in any other string in the log message.

{"name":"minio-dotnet : CopyObject_Test4", 
"function":"Task\u003CCopyObjectResult\u003E 
CopyObjectAsync(CopyObjectArgs args, 
CancellationToken cancellationToken = default(CancellationToken))",
"args":{"bucketName":"minio-dotnet-example-caeox1ekrstju5b",
"objectName":"d%a\u002B#c\u0026caa",
"destBucketName":"minio-dotnet-example-qsokc6g6es725dj",
"data":"1KB","size":"1KB"},
"duration":73,"status":"PASS"}

We need to turn the escaping off as they are just log messages.

martijn00 · 2023-04-06T21:07:38Z

@ebozduman I've done all of those things now. Can you have a look and merge?

martijn00 · 2023-04-07T20:23:02Z

@ebozduman Can you try again with the last commit?

martijn00 · 2023-04-10T10:49:51Z

@ebozduman finally a green check! Lets merge!

ebozduman

LGTM

martijn00 · 2023-04-11T18:52:40Z

@ebozduman Can you merge? I will follow up with some more warning cleanup afterwards.

ebozduman · 2023-04-11T20:28:48Z

@martijn00

Could you please rebase?

martijn00 · 2023-04-11T20:46:22Z

@ebozduman done the rebase now.

ebozduman · 2023-04-12T07:00:06Z

Hi @martijn00

looks like there are a little bit more to address. Thank you...

(cherry picked from commit 3363b25)

(cherry picked from commit 869fc5f)

martijn00 · 2023-04-12T07:29:28Z

@ebozduman fixed the merge conflicts now. Can you merge the PR?

martijn00 · 2023-04-13T07:04:02Z

@ebozduman thanks for merging! When do you expect to get a new nuget release published?

ebozduman · 2023-04-13T07:32:44Z

We'll try to do it this week.

* Start using hashing and span * Skip some tests for now * Working version * Cleanup * More span * Cleanup * Cleanup response * More Span * Fix hash * More span * Add netstandard compat back in * Fix null checks * Format * Fix all span for netstandard * Update package * Add net48 compat in tests * Fix warnings * Fix test * Completes changes to support 4.7.2 & 4.8 releases * Changes to support 4.x releases * Fix tags * Fix file name * turn the escaping off * Format * Update format * Fix tests * Update nuget * Fix warnings * More span * Format * Fix test * More span * Optimize streams * Format again * More memory * Format * Performance * Fix test * Progress * More stream * Format * Fix response * More span * Use memory * fixes the lint errors * Fix merge * Fixes (cherry picked from commit 3363b25) * More fixes (cherry picked from commit 869fc5f) * more lint changes --------- Co-authored-by: Ersan <ersan.bozduman@gmail.com>

martijn00 added 10 commits March 6, 2023 16:42

Start using hashing and span

3e4f77f

Skip some tests for now

04f4296

Working version

0cbb521

Cleanup

79d9919

More span

20a35df

Cleanup

1abb428

Cleanup response

273910d

More Span

5d9534b

Fix hash

b98d1e8

More span

313e061

ebozduman force-pushed the span-compat branch from 82e5efe to 525306e Compare April 4, 2023 23:41

ebozduman approved these changes Apr 10, 2023

View reviewed changes

Add netstandard compat back in

7541987

martijn00 and others added 18 commits April 11, 2023 22:44

Fix warnings

bfdcfce

More span

f77b71d

Format

053c757

Fix test

df30f1b

More span

90d003d

Optimize streams

58b1853

Format again

4ff299e

More memory

3f91559

Format

1bced1e

Performance

3778aef

Fix test

1ac6a11

Progress

1bc1e01

More stream

e7f1306

Format

de8d3f2

Fix response

645222f

More span

231579f

Use memory

cd2f4d1

fixes the lint errors

cecac37

martijn00 force-pushed the span-compat branch from 0ceb0fc to cecac37 Compare April 11, 2023 20:45

martijn00 added 3 commits April 12, 2023 09:28

Fix merge

b959ee2

Fixes

e8d12cd

(cherry picked from commit 3363b25)

More fixes

d05169b

(cherry picked from commit 869fc5f)

more lint changes

f13b642

ebozduman merged commit c43ef12 into minio:master Apr 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Span compat #757

Span compat #757

martijn00 commented Mar 23, 2023

ebozduman commented Mar 30, 2023

martijn00 commented Mar 30, 2023

martijn00 commented Mar 30, 2023

ebozduman commented Mar 30, 2023

martijn00 commented Mar 30, 2023

ebozduman commented Apr 3, 2023

ebozduman commented Apr 3, 2023

martijn00 commented Apr 3, 2023

ebozduman commented Apr 3, 2023

martijn00 commented Apr 4, 2023

ebozduman commented Apr 4, 2023 •

edited

Loading

ebozduman commented Apr 5, 2023

martijn00 commented Apr 6, 2023

martijn00 commented Apr 7, 2023

martijn00 commented Apr 10, 2023

ebozduman left a comment

martijn00 commented Apr 11, 2023

ebozduman commented Apr 11, 2023

martijn00 commented Apr 11, 2023

ebozduman commented Apr 12, 2023

martijn00 commented Apr 12, 2023

martijn00 commented Apr 13, 2023

ebozduman commented Apr 13, 2023

Span compat #757

Span compat #757

Conversation

martijn00 commented Mar 23, 2023

ebozduman commented Mar 30, 2023

martijn00 commented Mar 30, 2023

martijn00 commented Mar 30, 2023

ebozduman commented Mar 30, 2023

martijn00 commented Mar 30, 2023

ebozduman commented Apr 3, 2023

ebozduman commented Apr 3, 2023

martijn00 commented Apr 3, 2023

ebozduman commented Apr 3, 2023

martijn00 commented Apr 4, 2023

ebozduman commented Apr 4, 2023 • edited Loading

ebozduman commented Apr 5, 2023

martijn00 commented Apr 6, 2023

martijn00 commented Apr 7, 2023

martijn00 commented Apr 10, 2023

ebozduman left a comment

Choose a reason for hiding this comment

martijn00 commented Apr 11, 2023

ebozduman commented Apr 11, 2023

martijn00 commented Apr 11, 2023

ebozduman commented Apr 12, 2023

martijn00 commented Apr 12, 2023

martijn00 commented Apr 13, 2023

ebozduman commented Apr 13, 2023

ebozduman commented Apr 4, 2023 •

edited

Loading