You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 30, 2020. It is now read-only.
Recently I am doing some research in hadoop tools,and I tried white elephant. Everything is ok but uploading log files.
I tested it in my own computer with hadoop1.0.4. And my hadoop is in Pseudo-distributed.
My cfg.pm is below:
"hadoop_home" => "/home/libenchao/hadoop/hadoop-1.0.4"
"hadoop_logs" => "/home/libenchao/hadoop/hadoop-1.0.4/logs"
"days" => 5
"queues" => ['default']
"grid" => 'mygrid'
"destination" => "hdfs://localhost:9000/user/libenchao/history"
When I run: ./statsupload.pl --config cfg.pm
I get:
Checking the last 5 days in HDFS for existing data
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1130
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1129
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1128
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1127
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1126
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1125
Found 0 existing files in HDFS
Searching /home/libenchao/hadoop/hadoop-1.0.4/logs/history for logs
Found no new logs to upload
But this is the first time I upload files,and in my logs folder it does exist log files. I am confused if I am missing something.
I attach my log file folder below:
libenchao@libenchao-Aspire-4741:~/hadoop/hadoop-1.0.4/logs/history$ ls done/version-1/localhost_1385716167800_/2013/11/29/000000/
job_201311291709_0001_1385739836098_libenchao_word+count
job_201311291709_0001_conf.xml
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Recently I am doing some research in hadoop tools,and I tried white elephant. Everything is ok but uploading log files.
I tested it in my own computer with hadoop1.0.4. And my hadoop is in Pseudo-distributed.
My cfg.pm is below:
"hadoop_home" => "/home/libenchao/hadoop/hadoop-1.0.4"
"hadoop_logs" => "/home/libenchao/hadoop/hadoop-1.0.4/logs"
"days" => 5
"queues" => ['default']
"grid" => 'mygrid'
"destination" => "hdfs://localhost:9000/user/libenchao/history"
When I run: ./statsupload.pl --config cfg.pm
I get:
Checking the last 5 days in HDFS for existing data
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1130
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1129
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1128
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1127
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1126
/home/libenchao/hadoop/hadoop-1.0.4/bin/hadoop dfs -lsr hdfs://localhost:9000/user/libenchao/history/mygrid/daily/default/2013/1125
Found 0 existing files in HDFS
Searching /home/libenchao/hadoop/hadoop-1.0.4/logs/history for logs
Found no new logs to upload
But this is the first time I upload files,and in my logs folder it does exist log files. I am confused if I am missing something.
I attach my log file folder below:
libenchao@libenchao-Aspire-4741:~/hadoop/hadoop-1.0.4/logs/history$ ls done/version-1/localhost_1385716167800_/2013/11/29/000000/
job_201311291709_0001_1385739836098_libenchao_word+count
job_201311291709_0001_conf.xml
The text was updated successfully, but these errors were encountered: