Feature Request: Expected, Actual, and Because as Test Property Fields

[JustinGrote]

Summary of the feature request

The vscode proposed test UI has a very nice view window showing expected and actual results including a diff

I would like to be able to display this similar information from Pester, but currently since the error is just a string, it would require a lot of parsing.

How should it work? (optional)

Ideally a failure would add 3 parameters either to the data of the exception or to the testresult itself:
Example with Should -Be: Expected $false, but got $true

1. Expected - $false
2. Actual - $true

Example with Should -BeOfType: Expected the value to have type [int] or any of its subtypes, but got $true with type [bool].

1. Expected: [int]
2. Actual: [bool]

Also support Because, which would probably be shown as a comment to Expected

As a workaround a regex parsing of "Expected x, but y" would probably suffice for most scenarios

[nohwnd]

Sounds good. We already transport some data on the exception (in the data dictionary of the exception), this would be adding more items. We should make sure we save the serialized version of the object, to avoid holding references to the real object.

https://github.com/pester/Pester/blob/main/src/functions/assertions/Should.ps1#L17-L18

[fflaten]

IIRC that function isn't called anymore. It's been converted to a c# method in Pester.Factory.

Concept probably still applies though.

[nohwnd]

Yeah, the concept should be okay, the data are imho attached to the data dictionary on the exeception which is defined on the Exception base class so any .net exception has it.

[JustinGrote]

There is some information in targetObject but nothing specific to the error other than the message.

I think the most appropriate storage for this is $error.exception.data (currently blank), I'll go the parse route for now until this is available.

$x.errorrecord.targetobject

Key         Value
---         -----
Message     Expected $false, but got $true.
File        C:\Users\JGrote\Projects\vscode-pester-test-adapter\sample\Tests\Sample.tests.ps1
Line        5
LineText            It 'Context False' {$true | Should -be $false}
Terminating True

[ArmaanMcleod]

I'd be happy to PR this @nohwnd @JustinGrote.

A lot of the Should functions return a PSCustomObject with Succeeded/FailureMessage properties. Thinking we can can include Expected/Actual/Because here and adding these to the error record with the CreateShouldErrorRecord call inside Invoke-Assertion. Would also need to modify CreateErrorRecord to include these new properties inside the targetObject dictionary.

This would actually be very useful for the work I'm doing. I am also just parsing the message with regex and being able to access these properties inside the exception would be great.

[nohwnd]

Exception is a universal carrier of failure info. You can throw exceptions from your own assertions if you don't want to use Should. That is why I am suggesting to use Data on Exception object.

[JustinGrote]

6 month follow up :)

[nohwnd]

@JustinGrote hello from 4 more months into the future :D Someone needs to just implement this. Maybe you yourself? 😉

[JustinGrote]

@nohwnd thank you!

[nohwnd]

Thank you :)

[nohwnd]

It is now released in beta if you want to promote or try it further.

[fflaten]

@nohwnd The github release tag + notes are missing this + 3 other commits.

The published build is fine.

[nohwnd]

Thanks, I autogenerated the notes, not sure what went wrong. Odesláno z aplikace Outlook pro iOS<https://aka.ms/o0ukef>

…

________________________________ Od: Frode Flaten ***@***.***> Odesláno: Friday, April 26, 2024 7:50:59 AM Komu: pester/Pester ***@***.***> Kopie: Mention ***@***.***>; Comment ***@***.***>; State change ***@***.***>; Subscribed ***@***.***> Předmět: Re: [pester/Pester] Feature Request: Expected, Actual, and Because as Test Property Fields (Issue #1993) @nohwnd<https://github.com/nohwnd> The git tag incl. github release notes is missing this + 3 other commits. image.png (view on web)<https://github.com/pester/Pester/assets/3436158/8568153a-4fc3-41ca-ac83-25c1a8e6a1f3> The published build is fine though. — Reply to this email directly, view it on GitHub<#1993 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABLYLYP6BHWH6EOURUIXTQLY7HTMJBFKMF2HI4TJMJ2XIZLTSSBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVEZDAMJYGYZTKOJXURXGC3LFVFUGC427NRQWEZLMVRZXKYTKMVRXIX3UPFYGLLCJONZXKZKDN5WW2ZLOOSTHI33QNFRXHE4CUR2HS4DFVJZGK4DPONUXI33SPGSXMYLMOVS2OMJSGEZTINZUQKSHI6LQMWSWS43TOVS2K5TBNR2WLKJZGIYTANBQHEYDJAVEOR4XAZNFNRQWEZLMUV3GC3DVMWUTEMBRHA3DGNJZG6TXI4TJM5TWK4VGMNZGKYLUMU>. You are receiving this email because you were mentioned. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[nohwnd]

I've found only 2 commits missing, the tag was on wrong commit. not sure what other 2 I a missing, maybe those are the two for dropping powershell support and revert of that , which are linked at the bottom of the release notes?

[fflaten]

I think it was the azure-pipelines commits. But all good now, thanks 🙂

[nohwnd]

Yeah I think I made a commit locally into main, and then pulled because there was extra work, but instead of forwarding it merged, and I did not want that, so I reset to the remote state, cherry-pick the changes and push.

But before this I've tagged the latest commit, but that commit did not end up on main, because I cherry picked it.

And I do recall that I've reapplied the tag, but I probably did it wrong.