file.managed: contents_pillar and pillar.file_tree with keep_newline is broken in 2019.2.0

[tkaehn]

Description of Issue/Question

Using contents_pillar and pillar.file_tree with keep_newline is still broken in 2019.2.0 and results in removing lines.

Probably file.managed is broken as pillar.get shows the newlines (please see below).

Setup and steps to Reproduce Issue

root@minion:~# cat /etc/salt/minion.d/file_tree_pillar.conf
ext_pillar:
  - file_tree:
      root_dir: /srv/pillar/file_tree
      follow_dir_links: False
      keep_newline: True

root@minion:~# cat /srv/salt/motd.sls 
---
/etc/motd:
  file.managed:
    - contents_pillar: motd

root@minion:~# cp /srv/pillar/file_tree/hosts/minion/motd /etc/motd

root@minion:~# wc -l /srv/pillar/file_tree/hosts/minion/motd /etc/motd
 3 /srv/pillar/file_tree/hosts/minion/motd
 3 /etc/motd
 6 total

root@minion:~# salt-call --output=json pillar.get motd
{
    "local": "abc\n\n\n"
}

root@minion:~# salt-call state.apply motd test=False
local:
----------
          ID: /etc/motd
    Function: file.managed
      Result: True
     Comment: File /etc/motd updated
     Started: 13:12:56.206172
    Duration: 20.695 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,3 +1 @@
                   abc
                  -
                  -

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  20.695 ms

root@minion:~# wc -l /srv/pillar/file_tree/hosts/minion/motd /etc/motd
 3 /srv/pillar/file_tree/hosts/minion/motd
 1 /etc/motd
 4 total

Versions Report

root@minion:~# salt-call --versions-report
Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.8 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-8-amd64
         system: Linux
        version: debian 9.8 

[Ch3LL]

this should be fixed on the head of 2019.2 now and will be included in 2019.2.1. Any chance you can verify?

[tkaehn]

I checked out branch 2019.2 and it seems to work. However isn't it supposed to show a diff when running with test=True as well?

(2019.2) root@minion:/vagrant# wc -l /srv/pillar/file_tree/hosts/minion/motd /etc/motd
 3 /srv/pillar/file_tree/hosts/minion/motd
 1 /etc/motd
 4 total
(2019.2) root@minion:/vagrant# salt-call state.apply motd test=True
local:
----------
          ID: /etc/motd
    Function: file.managed
      Result: None
     Comment: The file /etc/motd is set to be changed
     Started: 08:44:02.451332
    Duration: 76.871 ms
     Changes:   

Summary for local
------------
Succeeded: 1 (unchanged=1)
Failed:    0
------------
Total states run:     1
Total run time:  76.871 ms
(2019.2) root@minion:/vagrant# salt-call state.apply motd test=False
local:
----------
          ID: /etc/motd
    Function: file.managed
      Result: True
     Comment: File /etc/motd updated
     Started: 08:44:08.973053
    Duration: 51.327 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1 +1,3 @@
                   abcä
                  +
                  +

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  51.327 ms
(2019.2) root@minion:/vagrant# wc -l /srv/pillar/file_tree/hosts/minion/motd /etc/motd
 3 /srv/pillar/file_tree/hosts/minion/motd
 3 /etc/motd
 6 total

[twangboy]

No, because there were no changes.

[tkaehn]

Please give me a short hint how to enable diff output as it was default in previous versions.

root@minion:/vagrant# . v2018.3.4/bin/activate
(v2018.3.4) root@minion:/vagrant# salt-call state.apply motd test=True
local:
----------
          ID: /etc/motd
    Function: file.managed
      Result: None
     Comment: The file /etc/motd is set to be changed
     Started: 08:25:43.129256
    Duration: 81.326 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,2 +1,3 @@
                   abcä
                   
                  +

Summary for local
------------
Succeeded: 1 (unchanged=1, changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  81.326 ms
(v2018.3.4) root@minion:/vagrant# deactivate 
root@minion:/vagrant# . 2019.2/bin/activate
(2019.2) root@minion:/vagrant# salt-call state.apply motd test=True
local:
----------
          ID: /etc/motd
    Function: file.managed
      Result: None
     Comment: The file /etc/motd is set to be changed
     Started: 08:25:57.632345
    Duration: 61.807 ms
     Changes:   

Summary for local
------------
Succeeded: 1 (unchanged=1)
Failed:    0
------------
Total states run:     1
Total run time:  61.807 ms

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.