视频1 视频21 视频41 视频61 视频文章1 视频文章21 视频文章41 视频文章61 推荐1 推荐3 推荐5 推荐7 推荐9 推荐11 推荐13 推荐15 推荐17 推荐19 推荐21 推荐23 推荐25 推荐27 推荐29 推荐31 推荐33 推荐35 推荐37 推荐39 推荐41 推荐43 推荐45 推荐47 推荐49 关键词1 关键词101 关键词201 关键词301 关键词401 关键词501 关键词601 关键词701 关键词801 关键词901 关键词1001 关键词1101 关键词1201 关键词1301 关键词1401 关键词1501 关键词1601 关键词1701 关键词1801 关键词1901 视频扩展1 视频扩展6 视频扩展11 视频扩展16 文章1 文章201 文章401 文章601 文章801 文章1001 资讯1 资讯501 资讯1001 资讯1501 标签1 标签501 标签1001 关键词1 关键词501 关键词1001 关键词1501 专题2001
hadoop2.5.2eclipse插件编译
2020-11-09 14:14:25 责编:小采
文档

最近在搞hadoop,所以编译hadoop-eclipse插件是少不了的 ! 先说下我的编译环境: hadoop: hadoop-2.5.2 OS:ubuntu 14.4 位(虚拟机) eclipse: eclipse-jee-luna-SR1-linux-gtk-x86_.tar.gz 1.下载源码: https://github.com/winghc/hadoop2x-eclipse-

最近在搞hadoop,所以编译hadoop-eclipse插件是少不了的 !

先说下我的编译环境:

hadoop: hadoop-2.5.2

OS:ubuntu 14.4 位(虚拟机)

eclipse: eclipse-jee-luna-SR1-linux-gtk-x86_.tar.gz

1.下载源码:

https://github.com/winghc/hadoop2x-eclipse-plugin

2.修改配置文件

libraries.properties

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#This properties file lists the versions of the various artifacts used by hadoop and components.
#It drives ivy and the generation of a maven POM
# This is the version of hadoop we are generating
hadoop.version=2.2.0
hadoop-gpl-compression.version=0.1.0

#These are the versions of our dependencies (in alphabetical order)
apacheant.version=1.7.0
ant-task.version=2.0.10

asm.version=3.2
aspectj.version=1.6.5
aspectj.version=1.6.11

checkstyle.version=4.2

commons-cli.version=1.2
commons-codec.version=1.4
=commons-collections.version=3.1
commons-configuration.version=1.6
commons-daemon.version=1.0.13
=commons-httpclient.version=3.0.1
=commons-lang.version=2.4
=commons-logging.version=1.0.4
commons-logging-api.version=1.0.4
=commons-math.version=2.1
commons-el.version=1.0
commons-fileupload.version=1.2
=commons-io.version=2.1
commons-net.version=3.1
core.version=3.1.1
coreplugin.version=1.3.2

hsqldb.version=1.8.0.10

ivy.version=2.1.0

jasper.version=5.5.12
=jackson.version=1.8.8
#not able to figureout the version of jsp & jsp-api version to get it resolved throught ivy
# but still declared here as we are going to have a local copy from the lib folder
jsp.version=2.1
jsp-api.version=5.5.12
jsp-api-2.1.version=6.1.14
jsp-2.1.version=6.1.14
jets3t.version=0.6.1
jetty.version=6.1.26
jetty-util.version=6.1.26
=jersey-core.version=1.8
=jersey-json.version=1.8
=jersey-server.version=1.8
=junit.version=4.5
jdeb.version=0.8
jdiff.version=1.0.9
json.version=1.0

kfs.version=0.1

log4j.version=1.2.15
lucene-core.version=2.3.1

mockito-all.version=1.8.5
jsch.version=0.1.42

oro.version=2.0.8

rats-lib.version=0.5.1

servlet.version=4.0.6
servlet-api.version=2.5
slf4j-api.version=1.4.3
slf4j-log4j12.version=1.4.3

wagon-http.version=1.0-beta-2
xmlenc.version=0.52
xerces.version=1.4.4

以上标"="的部分我修改为hadoop2.5.2所用的jar包版本

hadoop.version=2.5.2
commons-collections.version=3.2.1
commons-httpclient.version=3.1
commons-lang.version=2.6
commons-logging.version=1.1.3
commons-math.version=3.1.1
commons-io.version=2.4
jackson.version=1.9.13
jersey-core.version=1.9
jersey-json.version=1.9
jersey-server.version=1.9
junit.version=4.11


build.xml







 

 

 



 
 

 
 
 
 
 

 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 



 
 
 
 
 + 
 
 
 
 
 
 
 
 
 

 
 
	 
	 
 
 
 
 
 


以上以标有“=”的部分我修改为

 lib/commons-cli-${commons-cli.version}.jar,
 lib/commons-configuration-${commons-configuration.version}.jar,
 lib/commons-httpclient-${commons-httpclient.version}.jar,
 lib/commons-lang-${commons-lang.version}.jar,
 lib/jackson-core-asl-${jackson.version},
 lib/jackson-mapper-asl-${jackson.version}.jar,
 lib/slf4j-log4j12-${slf4j-log4j12.version}.jar,
 lib/slf4j-api-${slf4j-api.version}.jar,
标有“+”的部分是我加上的。

makePlus.sh

原文件内容

ant jar -Dversion=2.0.4 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/share/hadoop
修改后
ant jar -Dversion=2.5.2 -Declipse.home=/home/hadoop/eclipse -Dhadoop.home=/home/hadoop/hadoop-2.5.2

使之可以执行
chmod +X makePlus.sh

3.编译

进入makePlus.sh所在目录,执行我们修改好的makePlus.sh文件 ./makePlus.sh

在这过程中有一步ivy-resolve-common很长时间没反应,我不清楚是死掉了还是在运行中所以就一直等啊等,上网查说这一步是很花时间的我也就没有在意,

可是都一下午了还是没有任何提示成功或者失败。无奈,拿起qq搜了几个hadoop的群,进去问下,有人说慢很正常,网络卡顿了吧。我这一想有可能是网络不通,可是我的虚拟机是能联着网的,只有一种可能了,就是编译过程中需求去外国的网站下载所依赖的包,需要。想到这里我立马开个软件,再次执行 ./makePlus.sh,奇迹出现了!哈哈 编译成功了

Buildfile: /home/hadoop/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml

check-contrib:

init:
 [echo] contrib: eclipse-plugin

init-contrib:

ivy-download:
 [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
 [get] To: /home/hadoop/hadoop2x-eclipse-plugin/ivy/ivy-2.1.0.jar
 [get] Not modified - so not downloaded

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /home/hadoop/hadoop2x-eclipse-plugin/ivy/ivysettings.xml

ivy-resolve-common:
 [echo] loglevel:quiet;ant.project.name:eclipse-plugin

ivy-retrieve-common:
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = /home/hadoop/hadoop2x-eclipse-plugin/ivy/ivysettings.xml

compile:
 [echo] contrib: eclipse-plugin
 [javac] /home/hadoop/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds

jar:
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/jackson-core-asl-1.9.13.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/jackson-mapper-asl-1.9.13.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/slf4j-log4j12-1.7.5.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/slf4j-api-1.7.5.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/slf4j-api-1.7.5.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/guava-11.0.2.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/guava-11.0.2.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/hadoop-auth-2.5.2.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/hadoop-auth-2.5.2.jar
 [copy] Copying 1 file to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib
 [copy] Copying /home/hadoop/hadoop-2.5.2/share/hadoop/common/lib/netty-3.6.2.Final.jar to /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/lib/netty-3.6.2.Final.jar
 [jar] Building jar: /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.5.2.jar

BUILD SUCCESSFUL
Total time: 16 seconds

log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

[jar] Building jar: /home/hadoop/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.5.2.jar就是编译好的jar包所在路径了!

4.测试

把编译好的hadoop-eclipse-plugin-2.5.2.jar放在eclipse的plugins目录下,重启eclipse,配置好后发现不能 new hadoop location,现象是点击后没有任何反应。只好退出eclipse 在终端里进入eclipse 目录执行

./eclipse -clean -consolelog -debug
重新进入eclipse 点击 new hadoop location,查看终端里报出的异常信息java.lang.NoClassDefFoundError

经过一番百度之后build.xml添加绿色部分重新编译,放在eclipse的plugins目录下重启eclipse,便可以使用了

=

下载本文
显示全文
专题