file.managed fails with a list of one or more source files

[brianz]

My previously working states fail to run when upgrading from 2014.7.2 to 2014.7.4. The issue I'm seeing is that the source filed for file.managed must be a single item, rather than a list of one or more items.

From the docs:

http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.managed

A list of sources can also be passed in to provide a default source and a set of fallbacks. The first source in the list that is found to exist will be used and subsequent entries in the list will be ignored.

The following works:

rsyslog:
  file.managed:
    - name: /etc/rsyslog.conf
    - user: root
    - group: root
    - mode: 644 
    - source: salt://rsyslog/files/rsyslog.conf
    - require:
      - pkg: rsyslog

However, this fails:

rsyslog:
  file.managed:
    - name: /etc/rsyslog.conf
    - user: root
    - group: root
    - mode: 644
    - source:
      - salt://rsyslog/files/rsyslog.conf.{{ grains['fqdn'] }}
      - salt://rsyslog/files/rsyslog.conf.{{ grains['instance_type'] }}
      - salt://rsyslog/files/rsyslog.conf
    - require:
      - pkg: rsyslog

Changing to a single-item list still fails:

    - source:
      - salt://rsyslog/files/rsyslog.conf

Whether source has one item in the list or multiple, the error returned is:

"Comment: Unable to manage file: none of the specified sources were found"

As soon as I switch to the source: salt://... syntax, it works as expected.

$ salt --versions-report
           Salt: 2014.7.4
         Python: 2.6.9 (unknown, Apr  1 2015, 18:16:00)
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
 msgpack-python: 0.4.4
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 3.2.5
           Mako: Not Installed

[brianz]

After a WAG, it looks like this is a documentation bug. If sources is used rather than source, multiple items works.

[rallytime]

Thanks for the report and for updating your ticket. This should be a simple fix.

[lorengordon]

Interesting. I can't figure out how that's working. sources isn't a named variable in the function definition, and I don't see any reference to it in the function code. I figure sources is coming through in kwargs, but I must be missing which function is processing it...

[brianz]

I'm baffled too...b/c source was working just fine for us in the previous release. FWIW I'm using the gifts backend.

[brianz]

Ok...I'm mistaken...sorry. sources doesn't work. However, the state actually runs...it's just that it doesn't apply the change. So, my addendum is incorrect and the original issue still persists.

[rallytime]

@brianz Can you clarify what you meant in the last comment? The changes to all of the files you put in sources list actually occur, but the state run isn't reporting that correctly? Or the state "runs" but doesn't "fail" and no changes actually occur to the files in the sources list?

[brianz]

@rallytime ...sure.

So, original bug report description is valid.

When I changed the file.managed state to use sources

    - sources:
      - salt://rsyslog/files/rsyslog.conf.{{ grains['fqdn'] }}
      - salt://rsyslog/files/rsyslog.conf.{{ grains['instance_type'] }}
      - salt://rsyslog/files/rsyslog.conf

...note the s in sources...the state reported that it ran successfully. However, it either didn't run or ran incorrectly, because the state wasn't actually applied (i.e., the target file wasn't changed). When updating an existing file the existing file didn't get changed.

[jahamn]

@brianz I'm having trouble reproducing this issue. I've tested it on 2014.7.4, 2014.7.6, and 2015.5.3. Every one of these versions have worked with source being used as a list for me.

Are you still having the issue?
What OS are you using and how did you install salt?

[brianz]

I'll go back and test this again. IIRC Salt was installed via pip...this was on the latest Amazon Linux AMI.

[jfindlay]

Since this feature is documented to work already with source, I'm going to close this issue. If anyone has a compelling argument to move this behavior to a sources argument, we can reopen.

[ealphonse]

Hello all,

I also notice this regression behavior but only with a http source. I can reproduce the bug like this:

$ sudo cat /srv/salt/test.sls
installation:
  file.managed:
    - name: /tmp/test_file
    #- source: http://example.com/index.html
    - source:
      - http://example.com/index.html
      - salt://non_existent_file
    - source_hash: sha256=3587cb776ce0e4e8237f215800b7dffba0f25865cb84550e87ea8bbac838c423

calling the state with:

sudo salt-call --local state.sls test

And it works as a single line. Could someone confirm it is the right syntax?

$ salt-call --versions-report
Salt Version:
           Salt: 2015.8.8

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: 0.9.1
         PyYAML: 3.10
          PyZMQ: 14.0.1
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: 0.3.6
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-85-generic
         system: Ubuntu 14.04 trusty

[jfindlay]

@ealphonse, thanks for reporting. I can confirm this.

# cat /srv/salt/test.sls
installation:
  file.managed:
    - name: /tmp/bootstrap
    - source: https://bootstrap.saltstack.com/
    #- source:
    #  - https://bootstrap.saltstack.com/
    #  - salt://non_existent_file
    - source_hash: sha256=e4294157c2133ae78dbb0972ff22d86536ceffd35d57353e15094a43d6caffac

# salt jmoney-main state.apply test
jmoney-main:
----------
          ID: installation
    Function: file.managed
        Name: /tmp/bootstrap
      Result: True
     Comment: File /tmp/bootstrap updated
     Started: 13:09:08.328548
    Duration: 480.34 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644

Summary for jmoney-main
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1

# cat /srv/salt/test.sls
installation:
  file.managed:
    - name: /tmp/bootstrap
    #- source: https://bootstrap.saltstack.com/
    - source:
      - https://bootstrap.saltstack.com/
      - salt://non_existent_file
    - source_hash: sha256=e4294157c2133ae78dbb0972ff22d86536ceffd35d57353e15094a43d6caffac

jmoney-main ~ master # salt jmoney-main state.apply test
jmoney-main:
----------
          ID: installation
    Function: file.managed
        Name: /tmp/bootstrap
      Result: False
     Comment: Unable to manage file: none of the specified sources were found
     Started: 13:09:23.841516
    Duration: 26.519 ms
     Changes:

Summary for jmoney-main
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
ERROR: Minions returned with non-zero exit code

# salt --versions
Salt Version:
           Salt: 2015.8.8.2

Dependency Versions:
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
           Mako: 1.0.0
         PyYAML: 3.11
          PyZMQ: 14.4.0
         Python: 2.7.9 (default, Mar  1 2015, 12:57:24)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: 0.8.6
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.2
   mysql-python: Not Installed
      pycparser: 2.10
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: 0.8.2
        timelib: Not Installed

System Versions:
           dist: debian 8.4
        machine: x86_64
        release: 3.16.0-4-amd64
         system: debian 8.4

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[CaptainSanders]

It seems this problem had reared its head again. Version 2019.02 (master and minion) no longer accepts source when given multiple sources.

my.cnf:
file.managed:
- name: /etc/mysql/my.cnf
- source:
- source1
- source2

Gives this warning

      ID: my.cnf
Function: file.managed
    Name: /etc/mysql/my.cnf
  Result: False
 Comment: Unable to manage file: none of the specified sources were found

Yet changing the sls to:

my.cnf:
file.managed:
- name: /etc/mysql/my.cnf
- sources:
- source1
- source2

gives the following:

[WARNING ] State for file: /etc/mysql/my.cnf - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.

So now neither solution works. When the salt-master is 2018.03 (minion is still 2019.02) the first solution (source:) does work.