Add timeout option #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NHDaly
reviewed
Feb 15, 2023
| @@ -132,12 +132,20 @@ function get_free_testprocess(testitem, max_num_processes) | |||
| end | |||
| end | |||
|
|
|||
| function execute_test(test_process, testitem, testsetups) | |||
| function execute_test(test_process, testitem, testsetups, timeout) | |||
There was a problem hiding this comment.
@nickrobinson251 / @quinnj / @Drvi and I had been talking about possibly introducing a timeout parameter on the testitems themselves.
What do you think about on the testitem vs on the run_tests call? Maybe we'd want both?
There was a problem hiding this comment.
Yeah, I think both probably makes sense... The only thing I don't like about that is that timeouts generally seem somewhat machine specific (at least in some cases), and that doesn't seem to fit having them on the test items. But on the other hand, there are probably timeouts that are not of that nature where attaching them to a specific test item does make sense? Say a test that downloads something?
davidanthoff
added a commit
that referenced
this pull request
Jul 4, 2024
54d3970 Add support for filter callback 98be1ae Fix error reporting 31a012d Update README f70ab9b Update min bounds ab926ea Bump a min version a0ac63f Merge pull request #8 from julia-vscode:capture-output f633cfd Capture output 8c55c09 Add a TODO 1519678 Merge pull request #2 from julia-vscode:timeout cfede5d Add timeout option 7829a41 Add example 557f142 Export api 999d777 Fix a bug 2deebd2 Update Project.toml 389eb9f Merge commit '5891ea97cdda63931f7d911d9c779d00eaf4f3bf' 5891ea9 Squashed 'packages/TestItemServer/' changes from 2303253b..a31cd339 053086c Remove outdated stuff f0781af Remove task for TestItemDetection 86019d7 Remove TestItemDetection 5a9e907 Merge pull request #1 from julia-vscode:testsetup 1fb09c6 Add support for testsetup 15045f9 Squashed 'packages/TestItemServer/' changes from 943ad803..2303253b 1a43db8 Merge commit '15045f9e6353ee176580c570a0a822b224591f45' da61070 Remove obsolte subtrees 91518b2 Fix tests 3b2a798 Streamline code d15e779 Progress 04d52d1 Merge commit '854f65765723353f9863780e551424926e9804e0' 854f657 Squashed 'packages/TestItemDetection/' changes from 2c59c96c..a6689392 e7878ef Fix package detection 58d3200 Add an initial implementation e2986e3 Squashed 'packages/TestItemServer/' changes from ba3fb7ae..943ad803 5d660ba Merge commit 'e2986e3ded3712647c6c570e41d0217bc1edbf6a' a6400f5 Merge commit '2d33cf676f724ef477dd188980701ae81aa832b0' as 'packages/ProgressMeter' 2d33cf6 Squashed 'packages/ProgressMeter/' content from commit 83e619fa 48d64e0 Add update tasks 3ba1896 Add core structure 3e21e3a Squashed 'packages/TestItemServer/' content from commit ba3fb7ae a637fc0 Merge commit '3e21e3aaf5de4a7494afb460306b01589bec5491' as 'packages/TestItemServer' 86c682d Merge commit 'ec7d470c1ecd9b939821fcd6a1501a93ae1d8898' as 'packages/Revise' ec7d470 Squashed 'packages/Revise/' content from commit f5f8c6f1 6c050aa Merge commit '6febb6288f8837c14b2b8d3520716a614ce91a35' as 'packages/LoweredCodeUtils' 6febb62 Squashed 'packages/LoweredCodeUtils/' content from commit 1af0a400 7c14938 Merge commit '13830bf1b09b13f339c023dcc7cd9dd2b1da88f7' as 'packages/JuliaInterpreter' 13830bf Squashed 'packages/JuliaInterpreter/' content from commit b2ebd693 10279b4 Merge commit '7623b5194d6150d773f98dc6df6f7fbe7a805e91' as 'packages/JSONRPC' 7623b51 Squashed 'packages/JSONRPC/' content from commit c03664c 5456c81 Merge commit 'e58d29905db6446d461f947fff4a9a282574b46f' as 'packages/CodeTracking' e58d299 Squashed 'packages/CodeTracking/' content from commit c3195cd 6d418e5 Merge commit 'e825c17e17e35809bcfad7a424f655d5e6b0ed09' as 'packages/OrderedCollections' e825c17 Squashed 'packages/OrderedCollections/' content from commit afcc015 e64b3ae Merge commit '9a0feac9c9edd34e9a34ee3b6948dfc80df8aaef' as 'packages/JSON' 9a0feac Squashed 'packages/JSON/' content from commit 4b3913d cba9511 Merge commit 'd2b904c6f9a4650596b584333a9a34cc56ff18aa' as 'packages/URIParser' d2b904c Squashed 'packages/URIParser/' content from commit 3b90549 af7056c Merge commit 'b69e9bc1325881f372570c8201815276ccfc3d1f' as 'packages/TestEnv' b69e9bc Squashed 'packages/TestEnv/' content from commit 14dd0d9 449330b Merge commit 'f9b4d55828bf592328ceaf3acf7a3a3673535b30' as 'packages/Tokenize' f9b4d55 Squashed 'packages/Tokenize/' content from commit 8d2aa01 42850af Squashed 'packages/TestItemDetection/' content from commit 2c59c96 1540301 Merge commit '42850af80e5501db308ce93de723eba20312f977' as 'packages/TestItemDetection' 955755b Merge commit '5e14a36efe524bb2a77795ca17fda3e67f76a6a7' as 'packages/CSTParser' 5e14a36 Squashed 'packages/CSTParser/' content from commit 0d68ad7 3142930 Add empty readme REVERT: 14dd0d9 Convert to standard package layout REVERT: b8046cf Merge pull request #40 from kimlaberinto/fix-typo REVERT: 97ba0d7 Fix typo in README REVERT: 977633a Merge pull request #29 from kdheepak/patch-1 REVERT: 6551580 Fix typo in README REVERT: 1daede9 Merge pull request #25 from JuliaTesting/ox/cover14 REVERT: c2c1cd4 Change cover re:1.4 REVERT: 11cb5c6 Merge pull request #17 from JuliaTesting/ox/link REVERT: 2745986 Merge pull request #18 from nickrobinson251/patch-2 REVERT: c94b26e Remove user id from discourse link REVERT: 636cd00 Make example consistent REVERT: 101fb44 Cross link to the Discourse Release Announcement REVERT: 1735d11 Merge pull request #16 from nickrobinson251/patch-1 REVERT: 7989762 Add note on how to return to previous env REVERT: dac3736 Answer in readme: What should I put in my Project.toml `[compat]` section REVERT: 68d7457 Project.toml for tagbot to use REVERT: 6ab0496 Create TagBot.yml REVERT: e9ddd10 improve top exampke REVERT: 3290dca crosslink .2 REVERT: a9db410 Mention 1.3 branch REVERT: e6050a5 Make cover-sheet branch that has no code REVERT: a5ebfaa Merge pull request #6 from Wikunia/patch-1 REVERT: a8d2621 Typo an -> can REVERT: ce53a7b Merge pull request #1 from JuliaTesting/ox/doit REVERT: c84a676 1.7-nightly REVERT: 7f66a78 add 1.7 tests REVERT: 3756b34 add readme REVERT: 67a5f06 Restrict to 1.4+ (for now) REVERT: fe7b59d Make work in 1.7 REVERT: aa869e9 try and work on more versions of julia REVERT: afaffe4 Test test/project.toml REVERT: 4e33633 make work to just activate REVERT: ca43fab Bring over LICENSE REVERT: 9bcb6be redo everything REVERT: e2f33d2 initial setup REVERT: 7c15693 Tidy up imports REVERT: 199f711 Update runner command REVERT: c9c659a Fix test environment dependencies REVERT: 858db9e Refactor runner.jl REVERT: 6f5ae4c Refactor tests REVERT: e60e759 Make ReportingTestSet behaviour consistent REVERT: a7dc1e4 Modify runner to distinguish between exit codes REVERT: e197bdb Add GitHub Pages documentation and update docstrings REVERT: 2341207 Update have_color check to work with julia 1.5 REVERT: 2194840 Add logfilename argument REVERT: 5aa985f Make changes for code coverage REVERT: d28f51c Add test_args and julia_args to test() REVERT: 3ca38e8 Use sandboxes to load project and test dependencies REVERT: 578a88d Merge pull request #13 from mmiller-max/mm/console-output REVERT: 1e69dde Display normal test console output (#1) REVERT: cd417dc Merge pull request #15 from bauglir/fix-test-behavior-without-positional-arguments REVERT: 62eff55 Test active project when calling `test` without positional arguments REVERT: f0be51a Merge pull request #12 from bauglir/fix-runner-for-activated-projects REVERT: 3d79bba Fix runner for activated projects REVERT: dd60a4b Merge pull request #7 from mmiller-max/feature/version-1-compat REVERT: ac56c17 Change comments and formatting REVERT: 1c9d6d7 Remove debug logging and update reporttest.jl with Windows fix REVERT: 9db65e5 Use repr for paths REVERT: 4371021 Add more info statements REVERT: 238d28e Add info statements to debug windows CI REVERT: c7a6f79 Enable runner to work with all V1.0.5+ versions REVERT: a4c0a49 Stop printing out of ProcessFailedException REVERT: 32a7dc5 Add extra cmd inputs from Pkg.test REVERT: a906de3 Update runner REVERT: a0a4474 Remove old imports and update test REVERT: d195a0a Replace print_with_colour REVERT: 5c0ce2b =Separate out error checking and failing functionality REVERT: 4e301ac =fixed package run in cases of actually having test dependencies REVERT: 8f5a86c =added function to run on packages REVERT: 060bb52 initial empty commit git-subtree-dir: packages/TestEnv git-subtree-split: 54d3970
davidanthoff
added a commit
that referenced
this pull request
Jul 4, 2024
3d198c76 Correct version b1c2694e Bump version d65333ec Merge pull request #87 from kbarros/beta_fix b8f9be5e Try macOS-12 98d720d9 macOS-13 only 44ebdb72 Try caret on 1.11 0c1ceb1a Try tilde on 1.11 4d740b3f Add more Julia versions to test matrix 93bf66be Initial Julia 1.11 support 4d763795 Undo previous fix 46f5f730 Fix `sandbox` call on 1.11-beta 621a776e Update Project.toml b6d8e6c5 Update Project.toml 7ea1b5d4 Merge pull request #84 from KristofferC/patch-1 03981110 Remove the "absoluting stdlib paths" part ea550f24 Update Project.toml 72b6900e Merge pull request #83 from ericphanson/eph/support-reresolve a9aab696 fix tests on earlier versions 2b32c438 support re-resolve 0b9a63ea Update README.md 8a4d550c Update README.md 760eca36 Merge pull request #81 from jkrumbiegel/jkrumbiegel-patch-1 e7b94c4b Update README.md d57f7081 Update README.md 76cd8f55 Update Project.toml 7359b7ad Merge pull request #79 from maxkapur/version-number-prereleases a28a9af0 use `v"1.4-"` and similar with trailing dash in unit tests 81abc95c use e.g. `< v"1.4-"` instead of `< v"1.4"` to match versions in the 1.3 series 5adff7bf update authors ae622541 Set version to 1.100.0 6854fcf6 Fix version guard a91bf21d Fix typo 192bc0df Disable some more tests on specific platforms ff91579c Fix tests on MacOS 5fc24315 Add more CI targets 5fdfad0b Remove branch instructions from README 51436e42 Fix more test problems 2343af36 Split tests a bit by version 76786d3e Catch up with everything 309d31cc Catch up on 1.7 branch 25d9afb9 Catch up on 1.8/1.9 changes 323b5797 Use more compatible Pkg.add signature c521f70c Add CI e9461cde Add tests 14dd0d9a Convert to standard package layout b8046cf2 Merge pull request #40 from kimlaberinto/fix-typo 97ba0d7e Fix typo in README 977633af Merge pull request #29 from kdheepak/patch-1 65515800 Fix typo in README 1daede9a Merge pull request #25 from JuliaTesting/ox/cover14 c2c1cd4f Change cover re:1.4 11cb5c65 Merge pull request #17 from JuliaTesting/ox/link 27459868 Merge pull request #18 from nickrobinson251/patch-2 c94b26ea Remove user id from discourse link 636cd00a Make example consistent 101fb44c Cross link to the Discourse Release Announcement 1735d112 Merge pull request #16 from nickrobinson251/patch-1 79897621 Add note on how to return to previous env dac37363 Answer in readme: What should I put in my Project.toml `[compat]` section 68d74571 Project.toml for tagbot to use 6ab04962 Create TagBot.yml e9ddd101 improve top exampke 3290dca8 crosslink .2 a9db4105 Mention 1.3 branch e6050a5d Make cover-sheet branch that has no code a5ebfaa8 Merge pull request #6 from Wikunia/patch-1 a8d26212 Typo an -> can ce53a7b5 Merge pull request #1 from JuliaTesting/ox/doit c84a6767 1.7-nightly 7f66a78c add 1.7 tests 3756b343 add readme 67a5f063 Restrict to 1.4+ (for now) fe7b59d9 Make work in 1.7 aa869e90 try and work on more versions of julia afaffe48 Test test/project.toml 4e336333 make work to just activate ca43fab6 Bring over LICENSE 9bcb6be1 redo everything e2f33d2f initial setup 7c15693c Tidy up imports 199f7113 Update runner command c9c659a4 Fix test environment dependencies 858db9e3 Refactor runner.jl 6f5ae4c0 Refactor tests e60e759c Make ReportingTestSet behaviour consistent a7dc1e49 Modify runner to distinguish between exit codes e197bdbb Add GitHub Pages documentation and update docstrings 23412079 Update have_color check to work with julia 1.5 2194840e Add logfilename argument 5aa985fb Make changes for code coverage d28f51c4 Add test_args and julia_args to test() 3ca38e8d Use sandboxes to load project and test dependencies 578a88d4 Merge pull request #13 from mmiller-max/mm/console-output 1e69ddeb Display normal test console output (#1) cd417dcd Merge pull request #15 from bauglir/fix-test-behavior-without-positional-arguments 62eff554 Test active project when calling `test` without positional arguments f0be51a2 Merge pull request #12 from bauglir/fix-runner-for-activated-projects 3d79bbaa Fix runner for activated projects dd60a4b0 Merge pull request #7 from mmiller-max/feature/version-1-compat ac56c177 Change comments and formatting 1c9d6d79 Remove debug logging and update reporttest.jl with Windows fix 9db65e5c Use repr for paths 43710212 Add more info statements 238d28e8 Add info statements to debug windows CI c7a6f79f Enable runner to work with all V1.0.5+ versions a4c0a49c Stop printing out of ProcessFailedException 32a7dc52 Add extra cmd inputs from Pkg.test a906de36 Update runner a0a44748 Remove old imports and update test d195a0af Replace print_with_colour 5c0ce2b2 =Separate out error checking and failing functionality 4e301ac6 =fixed package run in cases of actually having test dependencies 8f5a86c9 =added function to run on packages 060bb52b initial empty commit REVERT: 54d3970 Add support for filter callback REVERT: 98be1ae Fix error reporting REVERT: 31a012d Update README REVERT: f70ab9b Update min bounds REVERT: ab926ea Bump a min version REVERT: a0ac63f Merge pull request #8 from julia-vscode:capture-output REVERT: f633cfd Capture output REVERT: 8c55c09 Add a TODO REVERT: 1519678 Merge pull request #2 from julia-vscode:timeout REVERT: cfede5d Add timeout option REVERT: 7829a41 Add example REVERT: 557f142 Export api REVERT: 999d777 Fix a bug REVERT: 2deebd2 Update Project.toml REVERT: 389eb9f Merge commit '5891ea97cdda63931f7d911d9c779d00eaf4f3bf' REVERT: 5891ea9 Squashed 'packages/TestItemServer/' changes from 2303253b..a31cd339 REVERT: 053086c Remove outdated stuff REVERT: f0781af Remove task for TestItemDetection REVERT: 86019d7 Remove TestItemDetection REVERT: 5a9e907 Merge pull request #1 from julia-vscode:testsetup REVERT: 1fb09c6 Add support for testsetup REVERT: 15045f9 Squashed 'packages/TestItemServer/' changes from 943ad803..2303253b REVERT: 1a43db8 Merge commit '15045f9e6353ee176580c570a0a822b224591f45' REVERT: da61070 Remove obsolte subtrees REVERT: 91518b2 Fix tests REVERT: 3b2a798 Streamline code REVERT: d15e779 Progress REVERT: 04d52d1 Merge commit '854f65765723353f9863780e551424926e9804e0' REVERT: 854f657 Squashed 'packages/TestItemDetection/' changes from 2c59c96c..a6689392 REVERT: e7878ef Fix package detection REVERT: 58d3200 Add an initial implementation REVERT: e2986e3 Squashed 'packages/TestItemServer/' changes from ba3fb7ae..943ad803 REVERT: 5d660ba Merge commit 'e2986e3ded3712647c6c570e41d0217bc1edbf6a' REVERT: a6400f5 Merge commit '2d33cf676f724ef477dd188980701ae81aa832b0' as 'packages/ProgressMeter' REVERT: 2d33cf6 Squashed 'packages/ProgressMeter/' content from commit 83e619fa REVERT: 48d64e0 Add update tasks REVERT: 3ba1896 Add core structure REVERT: 3e21e3a Squashed 'packages/TestItemServer/' content from commit ba3fb7ae REVERT: a637fc0 Merge commit '3e21e3aaf5de4a7494afb460306b01589bec5491' as 'packages/TestItemServer' REVERT: 86c682d Merge commit 'ec7d470c1ecd9b939821fcd6a1501a93ae1d8898' as 'packages/Revise' REVERT: ec7d470 Squashed 'packages/Revise/' content from commit f5f8c6f1 REVERT: 6c050aa Merge commit '6febb6288f8837c14b2b8d3520716a614ce91a35' as 'packages/LoweredCodeUtils' REVERT: 6febb62 Squashed 'packages/LoweredCodeUtils/' content from commit 1af0a400 REVERT: 7c14938 Merge commit '13830bf1b09b13f339c023dcc7cd9dd2b1da88f7' as 'packages/JuliaInterpreter' REVERT: 13830bf Squashed 'packages/JuliaInterpreter/' content from commit b2ebd693 REVERT: 10279b4 Merge commit '7623b5194d6150d773f98dc6df6f7fbe7a805e91' as 'packages/JSONRPC' REVERT: 7623b51 Squashed 'packages/JSONRPC/' content from commit c03664c REVERT: 5456c81 Merge commit 'e58d29905db6446d461f947fff4a9a282574b46f' as 'packages/CodeTracking' REVERT: e58d299 Squashed 'packages/CodeTracking/' content from commit c3195cd REVERT: 6d418e5 Merge commit 'e825c17e17e35809bcfad7a424f655d5e6b0ed09' as 'packages/OrderedCollections' REVERT: e825c17 Squashed 'packages/OrderedCollections/' content from commit afcc015 REVERT: e64b3ae Merge commit '9a0feac9c9edd34e9a34ee3b6948dfc80df8aaef' as 'packages/JSON' REVERT: 9a0feac Squashed 'packages/JSON/' content from commit 4b3913d REVERT: cba9511 Merge commit 'd2b904c6f9a4650596b584333a9a34cc56ff18aa' as 'packages/URIParser' REVERT: d2b904c Squashed 'packages/URIParser/' content from commit 3b90549 REVERT: af7056c Merge commit 'b69e9bc1325881f372570c8201815276ccfc3d1f' as 'packages/TestEnv' REVERT: b69e9bc Squashed 'packages/TestEnv/' content from commit 14dd0d9 REVERT: 449330b Merge commit 'f9b4d55828bf592328ceaf3acf7a3a3673535b30' as 'packages/Tokenize' REVERT: f9b4d55 Squashed 'packages/Tokenize/' content from commit 8d2aa01 REVERT: 42850af Squashed 'packages/TestItemDetection/' content from commit 2c59c96 REVERT: 1540301 Merge commit '42850af80e5501db308ce93de723eba20312f977' as 'packages/TestItemDetection' REVERT: 955755b Merge commit '5e14a36efe524bb2a77795ca17fda3e67f76a6a7' as 'packages/CSTParser' REVERT: 5e14a36 Squashed 'packages/CSTParser/' content from commit 0d68ad7 REVERT: 3142930 Add empty readme git-subtree-dir: packages/TestEnv git-subtree-split: 3d198c765a61ac08a779b802a307422ac202a9b1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requires julia-vscode/JSONRPC.jl#72.
The timeout is per testitem execution in seconds.