视频1 视频21 视频41 视频61 视频文章1 视频文章21 视频文章41 视频文章61 推荐1 推荐3 推荐5 推荐7 推荐9 推荐11 推荐13 推荐15 推荐17 推荐19 推荐21 推荐23 推荐25 推荐27 推荐29 推荐31 推荐33 推荐35 推荐37 推荐39 推荐41 推荐43 推荐45 推荐47 推荐49 关键词1 关键词101 关键词201 关键词301 关键词401 关键词501 关键词601 关键词701 关键词801 关键词901 关键词1001 关键词1101 关键词1201 关键词1301 关键词1401 关键词1501 关键词1601 关键词1701 关键词1801 关键词1901 视频扩展1 视频扩展6 视频扩展11 视频扩展16 文章1 文章201 文章401 文章601 文章801 文章1001 资讯1 资讯501 资讯1001 资讯1501 标签1 标签501 标签1001 关键词1 关键词501 关键词1001 关键词1501 专题2001
利用ycsb测试hbase性能
2020-11-09 07:47:07 责编:小采
文档


java 、maven、ycsb 的安装及配置见这篇博客: http://blog.csdn.net/hs794502825/article/details/17309845 本篇博客主要介绍 hbase 的安装,以及利用 ycsb 对 hbase 进行基本的测试 步骤一: 在http://mirrors.hust.edu.cn/apache/hbase/下载hbase稳定版:

java 、maven、ycsb 的安装及配置见这篇博客:
http://blog.csdn.net/hs794502825/article/details/17309845

本篇博客主要介绍 hbase 的安装,以及利用 ycsb 对 hbase 进行基本的测试

步骤一:
在http://mirrors.hust.edu.cn/apache/hbase/下载hbase稳定版:
hbase-0.94.14.tar.gz 存放于 /home/hs/program 目录下
cd /home/hs/program
tar -zxvf hbase-0.94.14.tar.gz


步骤二:
修改/home/hs/program/YCSB/hbase目录下的pom.xml文件(根据你所使用的hbase版本和hadoop版本,注意红色部分):


4.0.0

com.yahoo.ycsb
root
0.1.4

hbase-binding
HBase DB Binding


org.apache.hbase
hbase
${hbase.version}-->
0.94.14


org.apache.hadoop
hadoop-core
1.0.0-->
1.0.4


com.yahoo.ycsb
core
${project.version}







org.apache.maven.plugins
maven-assembly-plugin
${maven.assembly.version}


jar-with-dependencies

false



package

single











步骤三:
因为更改了步骤二中的文件,所以需要重新编译YCSB项目
cd /home/hs/program/YCSB
mvn clean package | tee mvn-clean-package.dat //将输出同时输出到屏幕和文件
显示与下类似信息,则说明编译成功:
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [7.540s]
[INFO] Core YCSB ............................................. SUCCESS [13.293s]
[INFO] Cassandra DB Binding .................................. SUCCESS [15.861s]
[INFO] HBase DB Binding ...................................... SUCCESS [2:21.346s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.296s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.347s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.140s]
[INFO] JDBC DB Binding ....................................... SUCCESS [5.495s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [4.054s]
[INFO] Mongo DB Binding ...................................... SUCCESS [3.368s]
[INFO] OrientDB Binding ...................................... SUCCESS [4.199s]
[INFO] Redis DB Binding ...................................... SUCCESS [3.261s]
[INFO] Voldemort DB Binding .................................. SUCCESS [4.113s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.7s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 minutes 8 seconds
[INFO] Finished at: Mon Dec 16 12:43:12 CST 2013
[INFO] Final Memory: 94M/227M

步骤四:
成功编译YCSB之后,在/home/hs/program/YCSB/distribution/target目录下会有ycsb-0.1.4.tar.gz
将该文件解压到/home/hs/program目录下:
tar -zxvf /home/hs/program/YCSB/distribution/target -C /home/hs/program

步骤五:
将/home/hs/program/hbase-0.94.14/lib中的以下文件:
slf4j-api-*.jar
slf4j-log4j12-*.jar
zookeeper-*.jar
拷贝到/home/hs/program/ycsb-0.1.4/hbase-binding/lib中

步骤六:
使用/home/hs/program/hbase-0.94.14/conf中的hbase-site.xml
替换掉/home/hs/program/ycsb-0.1.4/hbase-binding/conf中的hbase-site.xml

步骤七:
执行测试
终端1:
进入hbase的安装目录:
执行:./bin/start-hbase.sh
显示:
starting master, logging to /home/hs/program/hbase-0.94.14/bin/../logs/hbase-hs-master-hs-virtual-machine.out

接着执行:
./bin/hbase shell
create 'usertable','cf'

终端2中:
cd /home/hs/program/ycsb-0.1.4

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb load hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/load-hbase-a

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb run hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/run-hbase-a

得到如下结果:
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p hosts=localhost -p columnfamily=cf -t
[OVERALL], RunTime(ms), 5886.0
[OVERALL], Throughput(ops/sec), 169.466530750934
[UPDATE], Operations, 487
[UPDATE], AverageLatency(us), 9366.180698151951
[UPDATE], MinLatency(us), 33
[UPDATE], MaxLatency(us), 4459758
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 486
......
[READ], Operations, 514
[READ], AverageLatency(us), 2482.817120622568
[READ], MinLatency(us), 707
[READ], MaxLatency(us), 176125
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 29
[READ], Return=0, 514

下载本文
显示全文
专题