Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration test framework for isolated #3000

Merged
merged 7 commits into from
Jan 14, 2025

Conversation

andystaples
Copy link
Contributor

  • Add integration testing project
  • Methods for interacting with "func", "azurite", etc
  • One app with basic "Hello Cities" scenario
  • Build script that
    • installs a local copy of Core Tools
    • builds and deploys the extensions to the function app
    • builds the function app
    • installs and launches azurite

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation PR is ready to merge and referenced in pending_docs.md
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)
  • My changes do not require any extra work to be leveraged by OutOfProc SDKs
    • Otherwise: That work is being tracked here: #issue_or_pr_in_each_sdk
  • My changes do not change the version of the WebJobs.Extensions.DurableTask package
    • Otherwise: major or minor version updates are reflected in /src/Worker.Extensions.DurableTask/AssemblyInfo.cs
  • My changes do not add EventIds to our EventSource logs
    • Otherwise: Ensure the EventIds are within the supported range in our existing Windows infrastructure. You may validate this with a deployed app's telemetry. You may also extend the range by completing a PR such as this one.
  • My changes should be added to v3.x branch.
    • Otherwise: This change only applies to Durable Functions v2.x and will not be merged to branch v3.x.

- One app with basic "Hello Cities" scenario
@andystaples andystaples requested a review from cgillum December 18, 2024 19:43
@andystaples andystaples force-pushed the andystaples/isolated-integration-testing-framework branch from 2e4c00e to 9a68303 Compare December 18, 2024 19:50
Copy link
Member

@cgillum cgillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest a slightly revised file structure for this, basically putting all end-to-end related code under a test/e2e directory.

/test/e2e/Apps/DotNetIsolated/App.csproj
/test/e2e/Apps/DotNetIsolated/...
/test/e2e/Tests/E2ETests.csproj
/test/e2e/Tests/...

In the future, we can add other language-specific apps under the /test/e2e/Apps/ directory too.

Copy link
Collaborator

@bachuv bachuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening this PR!

test/DotnetIsolatedE2EApps/MainApp/app.csproj Outdated Show resolved Hide resolved
test/DotnetIsolatedE2ETests/Fixtures/FunctionAppFixture.cs Outdated Show resolved Hide resolved
test/DotnetIsolatedE2ETests/DotnetIsolatedE2ETests.csproj Outdated Show resolved Hide resolved
- Restructure e2e test folder structure
- Add e2e test readme/build instructions
- Move installed temp files to $env:TEMP or equivalent
- Do as many checks as possible before creating Process() object
Copy link
Member

@cgillum cgillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great. I've added some comments, but I don't think we necessarily need to block this PR on any of them.

test/e2e/Tests/Fixtures/FunctionAppFixture.cs Outdated Show resolved Hide resolved
test/e2e/Tests/Helpers/HttpHelpers.cs Outdated Show resolved Hide resolved
test/e2e/Tests/Helpers/HttpHelpers.cs Outdated Show resolved Hide resolved
var response = await GetResponseMessage(request);

Console.WriteLine(
$"InvokeHttpTrigger: {functionName}{queryString} : {response.StatusCode} : {response.ReasonPhrase}");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will console logs be captured in the test output? Normally we write logs in Xunit to ITestOutputHelper.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting this whole method as it is never used and - IMO - we should rely on the Task method and validate responses in tests or methods defined in test classes

string actualMessage = await response.Content.ReadAsStringAsync();
Console.WriteLine(
$"InvokeHttpTrigger: expectedMessage : {expectedMessage}, actualMessage : {actualMessage}");
return actualMessage.Contains(expectedMessage);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could/should we use Assert.Contains(...) here instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting this whole method as it is never used and - IMO - we should rely on the Task method and validate responses in tests or methods defined in test classes

<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.21.0" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="3.0.2" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason to include the WebJobs package reference here? Normally .NET Isolated apps should never reference WebJobs packages directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will address along with the work to update to package references

Copy link
Contributor Author

@andystaples andystaples Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - now the test app has a reference to the worker extension project only. The webjobs extension is still packed and put in a local nuget packages folder alongside the app, so that the WorkerExtensions.csproj can get the version it needs

test/e2e/Tests/e2e-tests-readme.md Show resolved Hide resolved
test/e2e/Tests/e2e-tests-readme.md Outdated Show resolved Hide resolved
test/e2e/Tests/start-emulators.ps1 Outdated Show resolved Hide resolved
.gitignore Outdated Show resolved Hide resolved
andystaples and others added 3 commits January 10, 2025 17:00
Direct commit applicable PR suggestions

Co-authored-by: Chris Gillum <cgillum@microsoft.com>
- file-scoped namespaces
- Remove unneeded .gitignore lines
- using statements for HTTP objects where applicable
- small readme improvements
Copy link
Member

@cgillum cgillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! As a follow up to this PR, let's create another PR to run these tests as part of the CI.

}

# Write-Host "Updating WebJobs.Extensions.DurableTask version to $webJobsExtensionVersion"
# dotnet add app.csproj package Microsoft.Azure.WebJobs.Extensions.DurableTask --version $webJobsExtensionVersion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can these commented-out lines be removed?

@andystaples andystaples merged commit 89b743e into dev Jan 14, 2025
6 checks passed
@andystaples andystaples deleted the andystaples/isolated-integration-testing-framework branch January 14, 2025 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants