testDelete fails intermittently

[fracture91]

Recently, my group has run into this test failing intermittently:

http://tiki.cs.wpi.edu:8080/jenkins/job/dragonsparkles4/72/testReport/

I ran some manual builds of our repo and got it to fail:

http://tiki.cs.wpi.edu:8080/jenkins/job/wpi-suite-tng/121/testReport/

This may be a symptom of the test database not being properly isolated from the "development" database. I feel like there's a chance that all groups are actually hitting the same database file on tiki, and one group's tests are now touching the database in a way that makes the tests fail. But that's just a hunch, I have no proof of that.

edu.wpi.cs.wpisuitetng.Db4oDatabaseTest.testDelete

expected:<{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}> but was:<null>

Stacktrace

junit.framework.AssertionFailedError: expected:<{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}> but was:<null>
    at edu.wpi.cs.wpisuitetng.Db4oDatabaseTest.testDelete(Db4oDatabaseTest.java:52)

Standard Output

Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}]
Stored {"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}
Stored {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}
retrievedAll: [{"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}, {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}]
retrievedAll: [{"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}]
retrievedAll: [{"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}]
Deleting: {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}
Stored {"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}
Stored {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}
retrievedAll: [{"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}, {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}]
retrievedAll: [{"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}, {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}]
Deleting: {"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}
Deleting: {"idNum":0,"username":"gafftron","name":"Gaffey","role":"USER"}
[]
[]
Stored {"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}
Stored {"name":"myProject","idNum":"0"}
retrievedAll: [{"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}]
Deleting: {"idNum":0,"username":"bgaffey","name":"Brian","role":"USER"}
retrievedAll: [{"name":"myProject","idNum":"0"}]
Deleting: {"name":"myProject","idNum":"0"}
retrievedAll: []
Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
Stored {"idNum":1,"username":"bgaffey","name":"Bryan","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}, {"idNum":1,"username":"bgaffey","name":"Bryan","role":"USER"}]
Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}]
Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}, {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}]
Stored {"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}
Stored {"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}, {"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}]
retrievedAll: [{"idNum":1,"username":"bgaffey","name":"Bryan","role":"USER"}, {"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}]
Deleting: {"idNum":1,"username":"bgaffey","name":"Bryan","role":"USER"}
Deleting: {"idNum":0,"username":"rchamer","name":"Mjolnir","role":"USER"}
Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
Stored {"idNum":1,"username":"rchamer","name":"Bryan","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}]
retrievedAll: [{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}, {"idNum":1,"username":"rchamer","name":"Bryan","role":"USER"}]
Deleting: {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
Deleting: {"idNum":1,"username":"rchamer","name":"Bryan","role":"USER"}
retrievedAll: []
Stored {"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}
Stored {"idNum":1,"username":"rchamer","name":"Bryan","role":"USER"}
Stored {"idNum":2,"username":"twack","name":"Tyler","role":"USER"}
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}, {"idNum":2,"username":"twack","name":"Tyler","role":"USER"}]
[{"idNum":0,"username":"rchamer","name":"Ryan","role":"USER"}]

Standard Error

Apr 04, 2013 8:13:30 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:31 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:32 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:32 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:32 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:35 PM edu.wpi.cs.wpisuitetng.database.DataStore update
INFO: Database Update Attempt...
Apr 04, 2013 8:13:35 PM edu.wpi.cs.wpisuitetng.database.DataStore update
INFO: Database Update Success!
Apr 04, 2013 8:13:36 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:37 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed
Apr 04, 2013 8:13:37 PM edu.wpi.cs.wpisuitetng.database.DataStore deleteAll
INFO: Database Delete All performed

[tuna-salad]

I've just been looking at this, too. The output is still similar to my team's first successful build. Although the test case only adds and then deletes a user "rchamer," I don't know why there is so much going on the standard output/error.

[fracture91]

I don't know why there is so much going on the standard output/error.

The DataStore implementation tends to output whatever it's doing, and I think the output Jenkins captured there includes previous test cases.

[tuna-salad]

So I think it means that DataStore is not behaving well and that the test results (especially testDelete) depend on the order of test case execution, where they shouldn't.

[Robrowski]

testUpdate in the same file (Db4oDatabaseTest.java) also fails. This comes up occasionally on Jenkins, but consistently comes up with testDelete failing as well when running just that file as a JUnit test. testUpdate fails because the user that is saved in as an ID of 0, but when pulled out the ID is 1.