我在配制nutch1.3,照着nutch的官方文档(http://wiki.apache.org/nutch/NutchTutorial)运行bin/nutch crawl urls -dir crawl -depth 3 -topN 5这一步的过程中会出现如下错误:
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124092006
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124091951
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124092038
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:175)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:149)
at org.apache.nutch.crawl.Crawl.run(Crawl.java:142)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:54)Hadoop log的提示是:
2011-11-24 09:21:08,939 WARN mapred.LocalJobRunner - job_local_0018
java.io.IOException: Cannot run program "chmod": java.io.IOException: error=12, Cannot allocate memory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:484)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:465)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:364)
at org.apache.hadoop.mapred.MapTask.localizeConfiguration(MapTask.java:111)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:173)最关键的一句是Cannot run program "chmod": java.io.IOException: error=12, Cannot allocate memory。它说不能分配内存,是我的内存(2G)太小了吗?我的操作系统是Ubuntu11.10,nutch的版本是1.3。我已经在网上找了很长时间,但都没有结果。请求大家的帮助。
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124092006
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124091951
LinkDb: adding segment: file:/home/zhou/nutch-1.3/runtime/local/crawl/segments/20111124092038
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:175)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:149)
at org.apache.nutch.crawl.Crawl.run(Crawl.java:142)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:54)Hadoop log的提示是:
2011-11-24 09:21:08,939 WARN mapred.LocalJobRunner - job_local_0018
java.io.IOException: Cannot run program "chmod": java.io.IOException: error=12, Cannot allocate memory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:484)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:465)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:364)
at org.apache.hadoop.mapred.MapTask.localizeConfiguration(MapTask.java:111)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:173)最关键的一句是Cannot run program "chmod": java.io.IOException: error=12, Cannot allocate memory。它说不能分配内存,是我的内存(2G)太小了吗?我的操作系统是Ubuntu11.10,nutch的版本是1.3。我已经在网上找了很长时间,但都没有结果。请求大家的帮助。
解决方案 »
- ssh2项目,表单提交到业务层时JSP报空指针错,请问都有哪些原因造成的?
- 如何获取海康威视DVR视频流
- 系统中基础信息库的处理
- c++中struct结构存储数据用java怎么读?
- 两个web工程这间的包调用问题
- java怎么调用存储过程
- 【菜鸟询问】servlet中为什么在继承了HttpServlet类之后,其实不能调用super.doGet()和super.doPost()方法呢
- 为什么org.hibernate.Session类没有find的方法呢?是不是版本不对的缘故呢
- 有没有人用java写过新浪微博客户端什么的?
- jboss like问题解决了/需要的朋友来看看
- 文件上传的路径问题
- 关于标签嵌套的写法
如果理解为内存不足的话虽然是2g,但如果未必是该程序可用内存的话...
和swap space memory有关吗,看过?
http://daimajishu.iteye.com/blog/959213
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:175)
at org.apache.nutch.crawl.LinkDb.invert(LinkDb.java:149)
at org.apache.nutch.crawl.Crawl.run(Crawl.java:142)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:54)这种程序不是跑在java虚拟机中的?那这些java代码是在哪里运行的
这个也会出异常?
你好。谢谢你的回复。我理解错你的意思了。我误以为你说的虚拟机是指在Windows下装一个Ubuntu的虚拟机。另外就是swap space memory太小的问题,我把swap space加到2G就好了。谢谢你。