Support log files on HDFS

[hercules90]

With a jobtracker HA deployment, it is a commonplace to use HDFS as a default location for storing jobtracker logs. Hence consuming logs which are directly stored in HDFS should be supported as well. As I see, it could be as simple as detecting the changed files similar to how it is done today and then simply copying over the changed files in the target structure.