我在CentOS 6.6上安装了一个基本的hadoop集群,并想编写一些基本程序(浏览文件系统,删除/添加文件等),但是我仍在努力使最基本的应用程序正常工作。
当运行一些基本代码以将目录的内容列出到控制台时,出现以下错误:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.ipc.RPC.getProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/hadoop/io/retry/RetryPolicy;Z)Lorg/apache/hadoop/ipc/VersionedProtocol; at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:135) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:280) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:245) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100) at mapreducetest.MapreduceTest.App.main(App.java:36)
我的pom.xml依赖项
<dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.6.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-core</artifactId> <version>1.2.1</version> </dependency> </dependencies>
代码:
import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileStatus; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.hdfs.DistributedFileSystem; public class App { public static void main( String[] args ) throws IOException, URISyntaxException { Configuration conf = new Configuration(); FileSystem fs = new DistributedFileSystem(); fs.initialize(new URI("hdfs://localhost:9000/"), conf); for (FileStatus f :fs.listStatus(new Path("/"))) { System.out.println(f.getPath().getName()); } fs.close(); } }
调用fs.initialize()后将引发错误。我真的不确定这是什么问题。我是否缺少依赖关系?他们是错误的版本吗?
我通过调用“ java -jar app.jar .... etc”来运行它,我应该一直在使用“ hadoop jar app.jar”。
当我正确运行它时,按预期工作。