下面内容是我执行命令输出的内容:
-----------------------------------------------------------------------------
[root@ikaixin01 hadoop-2.7.0]# hadoop jar /cloud/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount /words /count
Java HotSpot(TM) Client VM warning: You have loaded library /cloud/hadoop-2.7.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/06/02 02:19:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/02 02:19:14 INFO client.RMProxy: Connecting to ResourceManager at ikaixin01/192.168.137.88:8032
15/06/02 02:19:16 INFO input.FileInputFormat: Total input paths to process : 1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: number of splits:1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433170139587_0005
15/06/02 02:19:17 INFO impl.YarnClientImpl: Submitted application application_1433170139587_0005
15/06/02 02:19:17 INFO mapreduce.Job: The url to track the job: http://ikaixin01:8088/proxy/application_1433170139587_0005/
15/06/02 02:19:17 INFO mapreduce.Job: Running job: job_1433170139587_0005
15/06/02 02:19:28 INFO mapreduce.Job: Job job_1433170139587_0005 running in uber mode : false
15/06/02 02:19:28 INFO mapreduce.Job: map 0% reduce 0%
15/06/02 02:19:29 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_0, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000002 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:31 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_1, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000003 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:33 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_2, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000004 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:37 INFO mapreduce.Job: map 100% reduce 100%
15/06/02 02:19:38 INFO mapreduce.Job: Job job_1433170139587_0005 failed with state FAILED due to: Task failed task_1433170139587_0005_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:015/06/02 02:19:38 INFO mapreduce.Job: Counters: 4
Job Counters
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=0
Total time spent by all reduces in occupied slots (ms)=0
[root@ikaixin01 hadoop-2.7.0]#
------------------------------------------------------
网上查,都描述的是yarn-site.xml 缺少了aux-service的配置。
我的配置:
<property>
<name>yarn.nodemanager.aux-service</name>
<value>mapreduce_shuffle</value>
</property>麻烦大拿们帮忙看看是什么问题导致。
-----------------------------------------------------------------------------
[root@ikaixin01 hadoop-2.7.0]# hadoop jar /cloud/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount /words /count
Java HotSpot(TM) Client VM warning: You have loaded library /cloud/hadoop-2.7.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/06/02 02:19:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/02 02:19:14 INFO client.RMProxy: Connecting to ResourceManager at ikaixin01/192.168.137.88:8032
15/06/02 02:19:16 INFO input.FileInputFormat: Total input paths to process : 1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: number of splits:1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433170139587_0005
15/06/02 02:19:17 INFO impl.YarnClientImpl: Submitted application application_1433170139587_0005
15/06/02 02:19:17 INFO mapreduce.Job: The url to track the job: http://ikaixin01:8088/proxy/application_1433170139587_0005/
15/06/02 02:19:17 INFO mapreduce.Job: Running job: job_1433170139587_0005
15/06/02 02:19:28 INFO mapreduce.Job: Job job_1433170139587_0005 running in uber mode : false
15/06/02 02:19:28 INFO mapreduce.Job: map 0% reduce 0%
15/06/02 02:19:29 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_0, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000002 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:31 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_1, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000003 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:33 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_2, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000004 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)15/06/02 02:19:37 INFO mapreduce.Job: map 100% reduce 100%
15/06/02 02:19:38 INFO mapreduce.Job: Job job_1433170139587_0005 failed with state FAILED due to: Task failed task_1433170139587_0005_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:015/06/02 02:19:38 INFO mapreduce.Job: Counters: 4
Job Counters
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=0
Total time spent by all reduces in occupied slots (ms)=0
[root@ikaixin01 hadoop-2.7.0]#
------------------------------------------------------
网上查,都描述的是yarn-site.xml 缺少了aux-service的配置。
我的配置:
<property>
<name>yarn.nodemanager.aux-service</name>
<value>mapreduce_shuffle</value>
</property>麻烦大拿们帮忙看看是什么问题导致。
解决方案 »
- quantum-server 启动过程中卡住 且不报错
- 2012 OpenStack亚太技术大会 - 讲师 邓昳
- 创建了实例怎么进行端口转发呢?
- OpenStack兼容的华为云计算解决方案
- HP Helion帮你开启混和IT之门
- CentOS本地YUM安装单节点icehouse的异常
- LVS NAT模式,浏览器无法通过vip访问真实服务器的页面
- 如何学习超算
- aws dynamodb 存储引擎用的是什么策略?LSM吗?或者类似哪个开源数据库?
- Java中如何使用spark解析邮件
- CF login登陆报错 Invalid authentication token.
- 安装cf-uaac报错 ERROR: Failed to build gem native extension
<name>yarn.nodemanager.aux-services</name>编码要认真啊。