centos 6.6 hadoop 2.7.1 完全分布式安装部署

前端之家收集整理的这篇文章主要介绍了centos 6.6 hadoop 2.7.1 完全分布式安装部署前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

1.安装前,准备三台CENTOS 6.6系统的主机或虚机,并且关闭防火墙及selinux.

2.按如下表格配置IP地址,修改hosts文件及本机名

192.168.199.21hadoop21 Master

102.168.199.22hadoop22 Slave1

192.168.199.23hadoop23 Slave2


centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署
同理修改Slave1,Slave2的IP地址,hosts文件及本机名。

3.安装ORACLE JDK

先卸载本机openJDK,使用rpm -qa|grep java查看,然后用rpm -e 卸载

从oracle网站找到最新JDK,我这选择了JDK8

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

下载好以后解压,并移至/usr/java,如果没有可以mkdir 建立。

tar -xzvfjdk-8u51-linux-x64.gz

mvjdk1.8.0_51 /usr/java

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_51

export CLASSPATH=.:$JAVA_HOME/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export PATH=$PATH:$JAVA_HOME/bin

source /etc/profile


4.在各节点分别建立Hadoop运行帐号grid,并设置密码

5.配置SSH免密码登陆。

在各节点分别以grid用户名生成两个密钥文件,一个是私钥id_rsa,另一个是公钥id_rsa.pub

ssh-keygen -t rsa -f ~/.ssh/id_rsa

然后在hadoop21上

cp /home/grid/.ssh/id_rsa.pub /home/grid/.ssh/authorized_keys

scp hadoop22:/home/grid/.ssh/id_rsa.pub pubkeys22

scp hadoop23:/home/grid/.ssh/id_rsa.pub pubkeys23

cat pubkeys22 >> /home/grid/.ssh/authorized_keys

cat pubkeys23 >> /home/grid/.ssh/authorized_keys

rm pubkeys22

rm pubkeys23

最后分发authorized_keys到各节点

scp /home/grid/.ssh/authorized_keys hadoop22:/home/grid/.ssh

scp /home/grid/.ssh/authorized_keys hadoop23:/home/grid/.ssh

6.在Master机下载并解压Hadoop2.7.1(使用grid用户名)

找到最近的hadoop镜像,使用wget下载2.7.1

wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz

解压hadoop-2.7.1.tar.gz

tar -xzvfhadoop-2.7.1.tar.gz

cd hadoop-2.7.1

建立tmp,dfs,dfs/data,dfs/name

7.修改配置文件

修改hadoop-env.sh

export JAVA_HOME=/usr/java/jdk1.8.0_51


centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署


centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

8. 分发至各Salve节点

scp -r /home/grid/hadoop-2.7.1 hadoop22:/home/grid

scp -r /home/grid/hadoop-2.7.1 hadoop23:/home/grid

9.Master机格式化namenode

cd /home/grid/hadoop-2.7.1

./bin/hdfs namenode -format

10.启动Hadoop

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

11.验证是否成功

Master机应该启动NameNode,SecondaryNameNode,ResourceManager

Slave机应该启动Datanode,NodeManager

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

centos<wbr></p>6.6<wbr>hadoop<wbr>2.7.1<wbr>完全分布式安装部署

猜你在找的CentOS相关文章