-
准备机器
主机名 ip 对应的角色 master 192.168.202.5 NameNode
node1 192.168.202.6 Datanode1
node2 192.168.202.7 Datanode2 -
修改hosts
1. ###修改hostname 192.168.202.5 主机: echo "master" > /etc/hostname 192.168.202.6 主机: echo "node1" > /etc/hostname 192.168.202.7 主机: echo "node2" > /etc/hostname 2. ###master上修改hosts文件 echo "192.168.202.5 master" >>/etc/hosts echo "192.168.202.6 node1" >>/etc/hosts echo "192.168.202.7 node2" >>/etc/hosts 3. ###node修改同步到node1,node2上的主机 scp /etc/hosts node1:/etc/ scp /etc/hosts node2:/etc/ 4. ###各台主机上相互ping,测试是否可以通过host连通 ping master ping node1 ping node2 5. ###master,node1,node2上关闭防火墙 systemctl stop firewalld systemctl disable firewalld
-
配置免密码登陆
-
hadoop 安装(master)
下载 hadoop-2.7.3.tar.gz 把文件解压到 /usr/local/hadoop 修改环境变量 vi /etc/profile 环境变量(在master,node1,node2上都修改)
export JAVA_HOME=/usr/java/jdk1.8.0_65 export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:${HADOOP_HOME}/bin 关闭防火墙 service iptables stop
-
修改hadoop配置
master 服务器配置
-
vim /usr/local/hadoop/etc/hadoop/hadoop-env.sh
export JAVAHOME=/usr/java/jdk1.8.0121
-
vim /usr/local/hadoop/etc/hadoop/core-site.xml
<property> <name>hadoop.tmp.dir</name> <value>/hadoop/tmp</value> <description>Abase for other temporary directories.</description> </property> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>io.file.buffer.size</name> <value>4096</value> </property>
-
vim /usr/local/hadoop/etc/hadoop/hdfs-site.xml
<property> <name>dfs.replication</name> <value>2</value> <description>nodes total count</description> </property>
-
cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml
-
vi /usr/local/hadoop/etc/hadoop/mapred-site.xml
<property> <name>mapreduce.framework.name</name> <value>yarn</value> <final>true</final> </property> <property> <name>mapreduce.jobtracker.http.address</name> <value>master:50030</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>master:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>master:19888</value> </property> <property> <name>mapred.job.tracker</name> <value>http://master:9001</value> </property>
-
vi /usr/local/hadoop/etc/hadoop/yarn-site.xml
<property> <name>yarn.resourcemanager.hostname</name> <value>master</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>master:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>master:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>master:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>master:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>master:8088</value> </property>
-
vi /usr/local/hadoop/etc/hadoop/slaves
node1 node2
-
node1/node2服务器配置和master一样
scp -r /usr/local/hadoop node1:/usr/local scp -r /usr/local/hadoop node1:/usr/local
-
格式化master
cd /usr/local/hadoop ./hadoop namenode -format
-
启动hadoop(只在master启动)
cd /usr/local/hadoop/sbin ./start-all.sh 输入几个yes
-
查看进程 在master 上面 输入jps
24400 SecondaryNameNode 24565 ResourceManager 24678 NodeManager 24998 Jps 24200 Datanode 1391 Bootstrap 24063 NameNode
在node1和node2上输入jps
11408 Jps 11209 Datanode 1482 Bootstrap 11307 NodeManager
-
hadoop在web页面查看
http://192.168.202.5:8088/
参考文献
理论文档 http://www.cnblogs.com/edisonchou/p/3485135.html 安装教程 http://www.cnblogs.com/laov/p/3421479.html https://my.oschina.net/lanzp/blog/309078