-
Notifications
You must be signed in to change notification settings - Fork 601
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
At the time startLoading is called, canInitRequest should have assured that stub is != nil beforehand #51
Comments
Hi, Do you have a small/minimalist example project where you can reproduce this issue? As the assertion states, this seems illogical that Something related to multihreaded & race-conditions maybe? (like Anyway, an example project would help me a lot to debug this! |
In addition, did you try to |
I'll try to do it in I'm using Expecta on my project too, to test async behavior. |
Are you sure about that? Normally every Test framework correctly built (XCTest, OCUnit, GHUnit, etc…) execute its That's what makes |
I made a small test with XCTest (running tests inside Xcode) and that was the conclusion I figured out.
If I comment I'm new to |
i just wanted to chime in that I'm having the same exact problem, although I'm not trying to remove any stubs anywhere but my test suite teardown methods. I don't have time right now to write a reproducible project, but I'll try to make some in the next few days. I think you're right, that there's a race condition somewhere. I've got 4 test suites that all create their own stubs in the setup block, and remove them in the teardown. The last one gives me this null stub error. Something like
and the stub fails that same assert: 'Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'At the time startLoading is called, canInitRequest should have assured that stub is != nil beforehand''. My stacktrace is exactly the same as above, except line 5 is my code instead of his. Hopefully this helps a little... I'll see if I can get you something reproducible within a few days. I'm out of sprint time for bug resolutions for the week so I probably can't get to it until the weekend starts. Thanks for all the hard work, this is a great little library. |
@dreed1 Thanks for the feedback. I probably have an idea of the cause: do both of you ensure that the request has time to receive a response — or to timeout — before the end of your test cases? Especially, You should at least use something like |
I'm using Expecta's |
Damn so that's not the cause… Still interested by some code to reproduce it, understand what exactly you are doing and what can cause this… |
I think you were right about your idea: I have tests where I don't use However, I wasn't able to isolate it to a sample example. Even on my regular project it's hard to reproduce the issue (I'd say less than 1% of times running the full test suite). PS: I was also able to reproduce the issue when removing the stubs on |
Ok thanks for the feedback @marcelofabri So I'll start on the assumption that this only happens when you remove the stub after you sent the request but before waiting for the response to be returned (or the timeout to be exceeded), typically when you don't block your Test Case to wait for it (like when you forget to use Expecta's I'll try to secure all this a bit more anyway to avoid a crash in such case. Maybe just removing the assertion (and just returning directly without doing anything in If you have any suggestion on how I could detect and warn the user properly so that s/he realizes that s/he may have forgotten some active wait method (like the |
I see the same symptoms, but I see it maybe 25% of the time, which smells like a race condition, when running "all tests". |
@AliSoftware that turned out to be exactly the problem - I tear down the stub after each test, but some of the tests weren't waiting for the response to complete (because I didn't care about the result) which caused the race condition |
Just want to chime in here and say that I'm seeing the exact same behavior. Using Specta/Expecta, the |
Ok, so I should probably change the message of the assert so that it is a bit more explanatory, like:
I should even definitely write a wiki article at some point to explain this use case more thoroughly and how you should make your test cases wait for the response to arrive before the test can end and its status (success/failure) can then correctly determined. I would then put a link to this article in the assertion message to help users that encounter this. Will hopefully do that this weekend, but please remind me to do so if I didn't come up with this article past next week, I got a lot on my plate and I am able to forget about it 😉 |
fwiw, I had 0 synchronous expectations in my suite. Everything was using |
I never used Expecta, but shouldn't Maybe you could c/p a sample test of your Test Suite here to help me understand why you would hit the assertion in your specific case? |
You're exactly right about the expected behavior there. I couldn't figure out what the hell was going on either. Let me see if I can come up with a super simple reproduction when I get home, and I'll throw it up somewhere that you can get to. |
@AliSoftware Here's a test project I zipped up showing the exception. It happens really intermittently, you may have to run it 10 or more times. After pruning it down to the essentials, it hit the exception on the second run, but then I had to run it ~12 times to hit the next assertion. |
This last commit should fix it, by storing the stub in a ➡️ Version @gfontenot It retried your example project after applying this fix and wasn't able to reproduce the issue even after running your Unit Tests about 20 times. |
ping @gfontenot Is this still the issue you are dealing with on Twitter? Did you try to |
Nope, the more recent versions seem to have solved this particular issue. The issues I'm talking about on twitter seem to be completely unrelated to OHTTPStubs. Pretty much grasping at straws for those. |
…ace conditions (should hopefully fix #51)
I'm using
OHHTTPStubs
on my unit tests (usingXCTests
) and sometimes my tests fail because of an assert inOHHTTPStubs.m
(line 311).I don't know if it's related, but I remove all stubs (
[OHHTTPStubs removeAllStubs]
) before each test is run (it's implemented onsetUp
method in aXCTestCase
subclass that all my tests inherit from).Here's the stacktrace:
Any ideas?
The text was updated successfully, but these errors were encountered: