Flake: TestRebuild

[carolynvs]

I just ran an integration test against a new feature branch and saw the test fail in CI (but not locally of course)

2023-02-16T19:17:20.1480323Z === RUN   TestRebuild
2023-02-16T19:17:20.1480774Z     rebuild_test.go:24: porter create
2023-02-16T19:17:20.1489154Z 2023/02/16 19:03:53 exec: /tmp/porter729356652/porter porter create
2023-02-16T19:17:20.1490219Z     rebuild_test.go:44: porter explain --autobuild-disabled
2023-02-16T19:17:20.1492345Z 2023/02/16 19:03:53 exec: /tmp/porter729356652/porter porter explain --autobuild-disabled
2023-02-16T19:17:20.1493209Z     rebuild_test.go:49: porter explain
2023-02-16T19:17:20.1493739Z 2023/02/16 19:03:53 exec: /tmp/porter729356652/porter porter explain
2023-02-16T19:17:20.1494234Z     helpers.go:321: Editing porter.yaml
2023-02-16T19:17:20.1494867Z     rebuild_test.go:55: porter explain --autobuild-disabled -o=json
2023-02-16T19:17:20.1495601Z 2023/02/16 19:04:03 exec: /tmp/porter729356652/porter porter explain --autobuild-disabled -o=json
2023-02-16T19:17:20.1496114Z     helpers.go:67: 
2023-02-16T19:17:20.1496635Z         	Error Trace:	/tmp/porter-test3145471780/helpers.go:67
2023-02-16T19:17:20.1497236Z         	            				/tmp/porter-test3145471780/rebuild_test.go:56
2023-02-16T19:17:20.1498781Z         	Error:      	"{\n  \"name\": \"porter-hello\",\n  \"description\": \"An example Porter configuration\",\n  \"version\": \"0.1.0\",\n  \"porterVersion\": \"v1.0.8-9-g4ec3c74b\",\n  \"mixins\": [\n    \"exec\"\n  ]\n}\n" does not contain "WARNING: The bundle is out-of-date. Skipping autobuild because --autobuild-disabled was specified"
2023-02-16T19:17:20.1501721Z         	Test:       	TestRebuild
2023-02-16T19:17:20.1502106Z     helpers.go:69: {
2023-02-16T19:17:20.1502581Z           "name": "porter-hello",
2023-02-16T19:17:20.1503005Z           "description": "An example Porter configuration",
2023-02-16T19:17:20.1503414Z           "version": "0.1.0",
2023-02-16T19:17:20.1503885Z           "porterVersion": "v1.0.8-9-g4ec3c74b",
2023-02-16T19:17:20.1504517Z           "mixins": [
2023-02-16T19:17:20.1511386Z             "exec"
2023-02-16T19:17:20.1511746Z           ]
2023-02-16T19:17:20.1512066Z         }
2023-02-16T19:17:20.1512505Z         
2023-02-16T19:17:20.1512804Z         does not contain
2023-02-16T19:17:20.1513469Z         WARNING: The bundle is out-of-date. Skipping autobuild because --autobuild-disabled was specified
2023-02-16T19:17:20.1514058Z     main.go:178: Removing temp test PORTER_HOME
2023-02-16T19:17:20.1514634Z     main.go:181: Removing temp test directory
2023-02-16T19:17:20.1515123Z --- FAIL: TestRebuild (12.31s)

The bundle's porter.yaml is edited, but then we run porter explain with --autobuild-disabled which should cause us to see a warning, but otherwise the command runs successfully. We are not seeing the warning which means either the log level is wrong, or porter didn't detect that the bundle was out-of-date.

I don't have a good way to reproduce this at the moment so this is just an issue to track and help narrow down the cause if it happens again.