1.环境准备
想要更进一步了解和使用Trafodion,难免不得不自己尝试一番编译源码,此为之前做的一个小任务,踩过一些坑,现分享出来并做个总结。[参考](https://cwiki.apache.org/confluence/display/TRAFOdioN/Build+Source)
1.1 虚拟机
源码编译准备搭建自己的一个虚拟机,系统CentOS6.X即可,7版本会编译错误不支持。
配置参考:
硬盘:30G 最好不要默认,否则多次编译会使磁盘空间不足
内存:2G
网卡:NAT
Tips:避免出错需要重新新建虚拟机,最好每做一步做好快照,方便恢复。
查看系统版本:
cat /etc/redhat-release
CentOS release 6.7 (Final)
1.2 安装编译需要的包
# Install first to avoid dependency problems for the steps below
sudo yum install epel-release
sudo yum install alsa-lib-devel ant ant-nodeps apr-devel apr-util-devel \ boost-devel cmake device-mapper-multipath dhcp doxygen flex \ gcc-c++ gd git glibc-devel glibc-devel.i686 graphviz-perl gzip \ java-1.7.0-openjdk-devel libX11-devel libXau-devel libaio-devel \ libcurl-devel libibcm.i686 libibumad-devel libibumad-devel.i686 \ libiodbc libiodbc-devel librdmacm-devel librdmacm-devel.i686 \ libxml2-devel lua-devel lzo-minilzo \ net-snmp-devel net-snmp-perl openldap-clients openldap-devel \ openldap-devel.i686 openmotif openssl-devel openssl-devel.i686 \ perl-Config-IniFiles perl-Config-Tiny \ perl-DBD-sqlite perl-Expect perl-IO-Tty perl-Math-Calc-Units \ perl-Params-Validate perl-Parse-RecDescent perl-TermReadKey \ perl-Time-HiRes protobuf-compiler protobuf-devel \ readline-devel rpm-build saslwrapper sqlite-devel \ unixODBC unixODBC-devel uuid-perl wget xerces-c-devel xinetd
Tips: 避免因为包安装失败造成编译错误,请确保每一个包均安装成功
1.3 配置
配置java的环境变量,通过java -version 验证成功
mkdir trafodion
cd trafodion
git clone git://git.apache.org/incubator-trafodion.git
mkdir ~/trafodion-download ~/trafodion-build-tools
cd ~/mysource/incubator-trafodion/install
./traf_tools_setup.sh -d ~/trafodion-download -i ~/trafodion-build-tools
INFO: Starting tools build on Mon Jan 25 12:30:41 PST 2016
Tools install directory /home/trafdeveloper/trafodion-build-tools does not exist,do you want to to create it? y/n : y
INFO: Created directory /home/trafdeveloper/trafodion-build-tools
INFO: Tar download location: /home/trafdeveloper/download
INFO: Tool install directory location: /home/trafdeveloper/trafodion-build-tools
INFO: LogFile location: /home/trafdeveloper/mysource/incubator-trafodion/install/traf_tools_setup.log
***********************************************************
INFO: Installing MPI on Mon Jan 25 12:30:45 PST 2016
INFO: downloaded tar file: mpich-3.0.4.tar.gz
.
.
.
INFO: downloaded tar file: apache-maven-3.3.3-bin.tar.gz
INFO: Maven installation complete
***********************************************************
INFO: Completed tools build on Mon Jan 25 13:18:31 PST 2016
INFO: List of tools directory:
apache-log4cxx-0.10.0
apache-maven-3.3.3
bison_3_linux
dest-llvm-3.2
dest-mpich-3.0.4
icu4c_4.4
thrift-0.9.0
udis86-1.7.2
zookeeper-3.4.5
Tips : 此过程需要从网络上下载包并解压安装,由于网络限制,速度较慢,一般在1-1.5h
安装完成导入环境变量,后续需要使用,否则因为找不到变量而发生错误
export TOOLSDIR=$HOME/trafodion-build-tools
echo -en "export TOOLSDIR=$HOME/trafodion-build-tools\n" >> $HOME/.bashrc
本步骤结束,请验证
$JAVA_HOME
$TOOLSDIR
是否有值
2.编译
源码编译一般来说有debug和release两种方式,但是debug多用于开发百衲衣测试,在正式的环境中编译请推荐使用release方式
Tips:debug方式我没有编译成功过,具体问题需要待研究
d <Trafodion source directory>/core/sqf
source ./sqenvr.sh
cd ../..
执行 make all
或者make package-all(后续就不需要安装local_hadoop)
[INFO] Building jar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.jar
[INFO] Building tar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.tar.gz
[INFO] BUILD SUCCESS
mkdir -p ../distribution
mv target/dcs-1.3.0.tar.gz ../distribution
rm -f trafodion-dcs.jar.versions
make[2]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/dcs' ##(DCS) make[1]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/core'
Tips:这部分完成需要验证编译情况
sqvers -u
sqcers -u
TRAF_HOME=/home/trafdeveloper/mysource/incubator-trafodion/core/sqf
who@host=trafdeveloper@Box01
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.el6.x86_64
redhat=6.6
NO patches
Most common Apache_Trafodion Release 1.3.0 (Build debug [trafdeveloper],branch -,date 25Jan16)
UTT count is 1
[6] Release 1.3.0 (Build debug [trafdeveloper],date 25Jan16)
export/lib/hbase-trx-cdh5_3-1.3.0.jar
export/lib/hbase-trx-hbase_98_4-1.3.0.jar
export/lib/hbase-trx-hdp2_2-1.3.0.jar
export/lib/sqmanvers.jar
export/lib/trafodion-dtm-1.3.0.jar
export/lib/trafodion-sql-1.3.0.jar
3.搭建测试环境
编译完成,我们需要配置SSH免登录和测试一下环境服务
3.1 ssh 免登录
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh
3.2 修改默认配置
edit /etc/security/limits.conf and reboot the server
* - core 1000000
* - nproc 267263
* - nofile 32000
* - sigpending 515196
* - memlock 49595556
* - stack 10240
3.3 安装local_hadoop
如果前面编译make package-all,这步可不需要
d <Trafodion source directory>/core/sqf
source ./sqenvr.sh
cd ../..
cds
install_local_hadoop
Checking for existing Hadoop processes...
The testware tpcds_kit.zip does not exist and will not be installed
This testware is needed to run developer HIVE regression tests
.
.
.
Installed directory size and name = 2.7G /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/local_hadoop
Setup is complete. You can use the convenience scripts starting with sw... located in /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/scripts.
$ ./install_traf_components
Installing and configuring DCS,REST,TRAFCI & Phoenix tests for Trafodion...
Environment used for core,DCS,REST and Phonenix ...
.
.
.
Configuration scripts for DCS,TRAFCI and Phoenix test are set up
Open a new session and start Trafodion by executing sqgen and sqstart scripts
./install_traf_components
3.4 测试服务
测试服务状态
swstatus
# 无报错,打印两行字大致是所有服务以启动即可
# 一般HBase 没有正常启动
执行
swstarthbase 启动HMaster
#
sqgen
# 重新打开终端,引入源,执行
sqstart
所有服务启动正常,没有down的服务即大功告成
4.总结
1.以上Tips都是常遇到的错误,请注意
2.Trafodion 基于hadoop的基础
最后欢迎一起讨论学习Trafodion,支持企业版Esgyn