Flake: TestDesiredState #1978

carolynvs · 2022-03-23T16:01:18Z

I just saw TestDesiredState fail on canary because it couldn't find the outline line "The installation is out-of-sync, running the upgrade action". I feel like I have seen this before in CI so when we have time, let's figure out why it's flaking.

2022-03-23T14:33:47.8376812Z --- FAIL: TestDesiredState (127.22s)
2022-03-23T14:33:47.8379394Z     logger.go:17: waiting for the mongo service to be ready
2022-03-23T14:33:47.8380123Z         
2022-03-23T14:33:47.8381309Z     logger.go:17: Connecting to mongo database porter_TestDesiredState at [localhost:27017]
2022-03-23T14:33:47.8382692Z         
2022-03-23T14:33:47.8383843Z     logger.go:17: Dropping database porter_TestDesiredState!
2022-03-23T14:33:47.8385014Z         
2022-03-23T14:33:47.8385965Z     helpers.go:42: porter build --debug
2022-03-23T14:33:47.8387451Z     helpers.go:46: porter publish --reference localhost:5000/mydb:v0.1.0 --debug
2022-03-23T14:33:47.8388657Z     helpers.go:42: porter build --debug
2022-03-23T14:33:47.8389866Z     helpers.go:46: porter publish --reference localhost:5000/mybuns:v0.1.2 --debug
2022-03-23T14:33:47.8391402Z     helpers.go:30: porter parameters apply c:\a\2\s\tests\testdata\params\mybuns.yaml --namespace= --debug
2022-03-23T14:33:47.8393948Z     helpers.go:31: porter credentials apply c:\a\2\s\tests\testdata\creds\mybuns.yaml --namespace= --debug
2022-03-23T14:33:47.8395762Z     desiredstate_test.go:29: porter installation apply c:\a\2\s\tests\testdata\installations\invalid-schema.yaml --debug
2022-03-23T14:33:47.8397407Z     desiredstate_test.go:34: porter credentials apply c:\a\2\s\tests\testdata\creds\invalid-schema.yaml --debug
2022-03-23T14:33:47.8399359Z     desiredstate_test.go:39: porter parameters apply c:\a\2\s\tests\testdata\params\invalid-schema.yaml --debug
2022-03-23T14:33:47.8401023Z     desiredstate_test.go:44: porter parameters apply c:\a\2\s\tests\testdata\params\mybuns.yaml --namespace= --debug
2022-03-23T14:33:47.8402676Z     desiredstate_test.go:45: porter credentials apply c:\a\2\s\tests\testdata\creds\mybuns.yaml --namespace= --debug
2022-03-23T14:33:47.8405135Z     desiredstate_test.go:46: porter credentials apply c:\a\2\s\tests\testdata\creds\alt-mybuns.yaml --namespace= --debug
2022-03-23T14:33:47.8406838Z     helpers.go:116: Editing mybuns.yaml
2022-03-23T14:33:47.8408655Z     desiredstate_test.go:54: porter installation apply mybuns.yaml --namespace operator --debug
2022-03-23T14:33:47.8410236Z     helpers.go:116: Editing mybuns.yaml
2022-03-23T14:33:47.8412011Z     desiredstate_test.go:64: porter installation apply mybuns.yaml --namespace operator --debug
2022-03-23T14:33:47.8413989Z     helpers.go:50: porter show mybuns --namespace operator --output=json --debug
2022-03-23T14:33:47.8415397Z     desiredstate_test.go:71: porter installation apply mybuns.yaml --namespace operator --dry-run --debug
2022-03-23T14:33:47.8416921Z     desiredstate_test.go:76: porter installation apply mybuns.yaml --namespace operator --dry-run --force --debug
2022-03-23T14:33:47.8420556Z     helpers.go:116: Editing mybuns.yaml
2022-03-23T14:33:47.8422275Z     desiredstate_test.go:84: porter installation apply mybuns.yaml --namespace operator --debug
2022-03-23T14:33:47.8423443Z     helpers.go:116: Editing mybuns.yaml
2022-03-23T14:33:47.8424602Z     desiredstate_test.go:92: porter installation apply mybuns.yaml --namespace operator --debug
2022-03-23T14:33:47.8425641Z     helpers.go:69: 
2022-03-23T14:33:47.8426673Z         	Error Trace:	helpers.go:69
2022-03-23T14:33:47.8428032Z         	            				desiredstate_test.go:94
2022-03-23T14:33:47.8474795Z         	Error:      	"2022-03-23T14:32:39.165Z    debug    Writing logs to C:\\Users\\porterci\\AppData\\Local\\Temp\\porter587253642\\logs\\01FYVK39ZXRHSYVMAGHMD3G3RY.json
2022-03-23T14:32:39.175Z    debug    Writing logs to C:\\Users\\porterci\\AppData\\Local\\Temp\\porter587253642\\logs\\01FYVK39ZXRHSYVMAGHMD3G3RY.json
2022-03-23T14:32:39.175Z    debug    Reading input file mybuns.yaml
2022-03-23T14:32:39.175Z    debug    read input file    {\"contents\": \"schemaVersion: 1.0.0
name: mybuns
labels:
  generator: porter-operator
  generatorVersion: v0.2.0
  thing: 2
bundle:
  repository: localhost:5000/mybuns
  version: v0.1.2
credentialSets:
  - mybuns
parameterSets:
  - mybuns
uninstalled: false
parameters:
  log_level: 3
\"}
Connecting to mongo database porter_TestDesiredState at [localhost:27017]
Updated operator/mybuns installation
Reconciling operator/mybuns installation
Resolving parameter sources...
Resolving parameter source porter-state
2022-03-23T14:32:39.254Z    info    Triggering because the parameters have changed    {\"diff\": \"  map[string]string{
  \    \\\"cfg\\\":          \\\"\\\",
  \    \\\"chaos_monkey\\\": \\\"false\\\",
- \    \\\"log_level\\\":    \\\"11\\\",
+ \    \\\"log_level\\\":    \\\"3\\\",
  }
\"}
Injected installation operator/mybuns output porter-state as parameter porter-state
resolving parameters for installation operator/mybuns
Resolving parameter sources...
Resolving parameter source porter-state
Injected installation operator/mybuns output porter-state as parameter porter-state
resolving parameters for installation operator/mybuns
Resolving parameter sources...
Resolving parameter source porter-state
Injected installation operator/mybuns output porter-state as parameter porter-state
2022-03-23T14:32:39.276Z    debug    Using runtime driver docker

saving action upgrade for operator/mybuns installation with status running
params:
  - cfg: 
  - chaos_monkey: false
  - log_level: 3
  - porter-debug: true
  - porter-state: H4sIAAAAAAAA/+zRTQoDIQwF4Kx7Ci9Q+hSJxykitgitFs3cv9Af6Ga6GplNvo1IsniPPFqX3I9DouTTPV5LOl/KLdOWAIC9J9gQQvCvP4DvCzgm651ldoFhCZYdMxlsmmLFMiR2AmqrvTVZ3fs7/JT8KTUj6gzv+6diyjDRLLWk1uth71RKKaVmewYAAP//4RNJPQAIAAA=
creds:
  - username: *******

DEBUG: defaulting action to CNAB_ACTION (upgrade)
executing upgrade action from mybuns (installation: operator/mybuns)
Unpacking bundle state...
  - magic_file -> magic.txt
=== Step Data ===
map[bundle:map[credentials:map[username:*******] custom:map[] dependencies:map[] description:A very thorough test bundle images:map[] invocationImage:localhost:5000/mybuns:332dd75c541511a27fc332bdcd049d5b name:mybuns outputs:map[] parameters:map[cfg:buncfg.json chaos_monkey:false log_level:3] version:0.1.2] env:map[CHAOS_MONKEY:false CNAB_ACTION:upgrade CNAB_BUNDLE_NAME:mybuns CNAB_BUNDLE_VERSION:0.1.2 CNAB_CLAIMS_VERSION:1.0.0-DRAFT+b5ed2f3 CNAB_INSTALLATION_NAME:operator/mybuns CNAB_REVISION:01FYVK3A373FYQPQZ9JQ7HSBNW CUSTOM_VAR:boop HOME:/home/nonroot HOSTNAME:a3bc31e98a25 LOG_LEVEL:3 PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PORTER_DEBUG:true PORTER_HOME:/cnab/app PWD:/cnab/app SHLVL:0 USERNAME:*******] installation:map[name:operator/mybuns]]
=== Step Template ===
exec:
  arguments:
    - ensureMagic
  command: ./helpers.sh
  description: Ensure magic

=== Rendered Step ===
exec:
  arguments:
    - ensureMagic
  command: ./helpers.sh
  description: Ensure magic

Ensure magic
DEBUG name:    exec
DEBUG pkgDir: /cnab/app/mixins/exec
DEBUG file:     
DEBUG stdin:
upgrade:
  - exec:
      arguments:
        - ensureMagic
      command: ./helpers.sh
      description: Ensure magic

/cnab/app/cnab/app/mixins/exec/runtimes/exec-runtime upgrade --debug
DEBUG Parsed Input:
&exec.Action{Name:\"upgrade\", Steps:[]exec.Step{exec.Step{Instruction:exec.Instruction{Description:\"Ensure magic\", Command:\"./helpers.sh\", WorkingDir:\"\", Arguments:[]string{\"ensureMagic\"}, SuffixArguments:[]string(nil), Flags:builder.Flags(nil), EnvironmentVars:map[string]string(nil), Outputs:[]exec.Output(nil), SuppressOutput:false, IgnoreErrorHandler:builder.IgnoreErrorHandler{All:false, ExitCodes:[]int(nil), Output:builder.IgnoreErrorWithOutput{Contains:[]string(nil), Regex:[]string(nil)}}}}}}
/cnab/app ./helpers.sh ensureMagic
=== Step Data ===
map[bundle:map[credentials:map[username:*******] custom:map[] dependencies:map[] description:A very thorough test bundle images:map[] invocationImage:localhost:5000/mybuns:332dd75c541511a27fc332bdcd049d5b name:mybuns outputs:map[] parameters:map[cfg:buncfg.json chaos_monkey:false log_level:3] version:0.1.2] env:map[CHAOS_MONKEY:false CNAB_ACTION:upgrade CNAB_BUNDLE_NAME:mybuns CNAB_BUNDLE_VERSION:0.1.2 CNAB_CLAIMS_VERSION:1.0.0-DRAFT+b5ed2f3 CNAB_INSTALLATION_NAME:operator/mybuns CNAB_REVISION:01FYVK3A373FYQPQZ9JQ7HSBNW CUSTOM_VAR:boop HOME:/home/nonroot HOSTNAME:a3bc31e98a25 LOG_LEVEL:3 PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PORTER_DEBUG:true PORTER_HOME:/cnab/app PWD:/cnab/app SHLVL:0 USERNAME:*******] installation:map[name:operator/mybuns]]
=== Step Template ===
exec:
  arguments:
    - upgrade
  command: ./helpers.sh
  description: upgrade
  outputs:
    - name: mylogs
      regex: (.*)

=== Rendered Step ===
exec:
  arguments:
    - upgrade
  command: ./helpers.sh
  description: upgrade
  outputs:
    - name: mylogs
      regex: (.*)

DEBUG name:    exec
DEBUG pkgDir: /cnab/app/mixins/exec
DEBUG file:     
DEBUG stdin:
upgrade:
  - exec:
      arguments:
        - upgrade
      command: ./helpers.sh
      description: upgrade
      outputs:
        - name: mylogs
          regex: (.*)

/cnab/app/cnab/app/mixins/exec/runtimes/exec-runtime upgrade --debug
upgrade
DEBUG Parsed Input:
&exec.Action{Name:\"upgrade\", Steps:[]exec.Step{exec.Step{Instruction:exec.Instruction{Description:\"upgrade\", Command:\"./helpers.sh\", WorkingDir:\"\", Arguments:[]string{\"upgrade\"}, SuffixArguments:[]string(nil), Flags:builder.Flags(nil), EnvironmentVars:map[string]string(nil), Outputs:[]exec.Output{exec.Output{Name:\"mylogs\", FilePath:\"\", JsonPath:\"\", Regex:\"(.*)\"}}, SuppressOutput:false, IgnoreErrorHandler:builder.IgnoreErrorHandler{All:false, ExitCodes:[]int(nil), Output:builder.IgnoreErrorWithOutput{Contains:[]string(nil), Regex:[]string(nil)}}}}}}
/cnab/app ./helpers.sh upgrade
Processing regex output mylogs...roll the dice with your chaos monkey
=== Step Data ===
map[bundle:map[credentials:map[username:*******] custom:map[] dependencies:map[] description:A very thorough test bundle images:map[] invocationImage:localhost:5000/mybuns:332dd75c541511a27fc332bdcd049d5b name:mybuns outputs:map[mylogs:] parameters:map[cfg:buncfg.json chaos_monkey:false log_level:3] version:0.1.2] env:map[CHAOS_MONKEY:false CNAB_ACTION:upgrade CNAB_BUNDLE_NAME:mybuns CNAB_BUNDLE_VERSION:0.1.2 CNAB_CLAIMS_VERSION:1.0.0-DRAFT+b5ed2f3 CNAB_INSTALLATION_NAME:operator/mybuns CNAB_REVISION:01FYVK3A373FYQPQZ9JQ7HSBNW CUSTOM_VAR:boop HOME:/home/nonroot HOSTNAME:a3bc31e98a25 LOG_LEVEL:3 PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PORTER_DEBUG:true PORTER_HOME:/cnab/app PWD:/cnab/app SHLVL:0 USERNAME:*******] installation:map[name:operator/mybuns]]
=== Step Template ===
exec:
  arguments:
    - chaos_monkey
    - '{{ bundle.parameters.chaos_monkey }}'
  command: ./helpers.sh
  description: roll the dice with your chaos monkey

=== Rendered Step ===
exec:
  arguments:
    - chaos_monkey
    - 'false'
  command: ./helpers.sh
  description: roll the dice with your chaos monkey

DEBUG name:    exec
DEBUG pkgDir: /cnab/app/mixins/exec
DEBUG file:     
DEBUG stdin:
upgrade:
  - exec:
      arguments:
        - chaos_monkey
        - \"false\"
      command: ./helpers.sh
      description: roll the dice with your chaos monkey

/cnab/app/cnab/app/mixins/exec/runtimes/exec-runtime upgrade --debug
DEBUG Parsed Input:
&exec.Action{Name:\"upgrade\", Steps:[]exec.Step{exec.Step{Instruction:exec.Instruction{Description:\"roll the dice with your chaos monkey\", Command:\"./helpers.sh\", WorkingDir:\"\", Arguments:[]string{\"chaos_monkey\", \"false\"}, SuffixArguments:[]string(nil), Flags:builder.Flags(nil), EnvironmentVars:map[string]string(nil), Outputs:[]exec.Output(nil), SuppressOutput:false, IgnoreErrorHandler:builder.IgnoreErrorHandler{All:false, ExitCodes:[]int(nil), Output:builder.IgnoreErrorWithOutput{Contains:[]string(nil), Regex:[]string(nil)}}}}}}
/cnab/app ./helpers.sh chaos_monkey false
Collecting bundle outputs...
Packing bundle state...
  - magic.txt
Closing plugins
execution completed successfully!
Closing plugins
" does not contain "The installation is out-of-sync, running the upgrade action"

The text was updated successfully, but these errors were encountered:

carolynvs · 2022-09-15T19:24:02Z

Closing as I haven't seen this in months

carolynvs added the meta Things necessary to make this project run day to day label Mar 23, 2022

carolynvs closed this as completed Sep 15, 2022

schristoff mentioned this issue Sep 13, 2023

(epic) Flaky Tests #2906

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flake: TestDesiredState #1978

Flake: TestDesiredState #1978

carolynvs commented Mar 23, 2022 •

edited

Loading

carolynvs commented Sep 15, 2022

Flake: TestDesiredState #1978

Flake: TestDesiredState #1978

Comments

carolynvs commented Mar 23, 2022 • edited Loading

carolynvs commented Sep 15, 2022

carolynvs commented Mar 23, 2022 •

edited

Loading