http://blog.csdn.net/circyo/article/details/46724335
CentOS7.0安装配置hadoop2.7.0
资源准备
资源下载:
- hadoop-2.7.0.tar.gz密码:727y
- jdk-8u45-linux-x64.tar.gz密码:d8bm
- CentOS-7.0-1406-x86_64-DVD.iso密码:1a71
- virtual box for mac密码:h9ig
注意事项:
- 如果自己下载资源的话,注意hadoop,jdk,centos都应该是64位或者32位的,以免出现无法预料的错误,上面的资源都是64位的
- 我是在mac下配置的,virtual Box是ios x系统的,如果是其它系统的另外寻找资源下载安装
linux 虚拟机配置
系统配置:
- 虚拟机:一个master(Master.Hadoop),两个slave(Slave1.Hadoop,Slave2.Hadoop)
- 网络设置:我使用的是桥接的方式,因为只是实验性安装,我没有设置静态ip
- 内存:每个虚拟机配置1024M内存
- 分区:自动
- 软件选择:最小安装,注意选择开发工具
- 用户设置:密码都设置为:hadoophadoop,不创建任何用户,操作时使用root直接进行
额外软件安装:
- centos7.0初始化时并没有ifconfig指令,需要使用下面两个指令进行安装:
完成其它两个虚拟机的安装:
- 两个slave的hostname可以改成Slave1.Hadoop,Slave2.Hadoop,方便区分
安装完后各个虚拟机的ip配置(参考用)
主机 | ip地址 |
---|---|
master.hadoop | 192.168.1.122 |
slave1.hadoop | 192.168.1.125 |
slave2.hadoop | 192.168.1.124 |
配置本地hosts
- 输入指令
- 使用以下指令对master主机中进行测试,可使用类似指令在slave主机测试
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">ping</span> Slave1<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.Hadoop</span> <span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">ping</span> Slave2<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.Hadoop</span> </code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li></ul>
配置Master无密码登录所有Salve
以下在Master主机上配置
- 输入以下指令生成ssh
- 把id_rsa.pub追加到授权的key里面去
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">cat ~/<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.ssh</span>/id_rsa<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.pub</span> >> ~/<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.ssh</span>/authorized_keys</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul>
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">chmod <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">600</span> ~<span class="hljs-regexp" style="color: rgb(0,0); Box-sizing: border-Box;">/.ssh/authorized</span>_keys</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul>
- 设置SSH配置
- 重启ssh服务
- 把公钥复制所有的Slave机器上
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">/<span class="hljs-regexp" style="color: rgb(0,0); Box-sizing: border-Box;">/ scp ~/</span>.ssh/id_rsa.pub 远程用户名@远程服务器<span class="hljs-constant" style="Box-sizing: border-Box;">IP</span><span class="hljs-symbol" style="color: rgb(0,102); Box-sizing: border-Box;">:~/</span> scp ~<span class="hljs-regexp" style="color: rgb(0,0); Box-sizing: border-Box;">/.ssh/id</span>_rsa.pub root<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">@192</span>.<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">168.1</span>.<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">125</span><span class="hljs-symbol" style="color: rgb(0,102); Box-sizing: border-Box;">:~/</span> scp ~<span class="hljs-regexp" style="color: rgb(0,102); Box-sizing: border-Box;">124</span><span class="hljs-symbol" style="color: rgb(0,102); Box-sizing: border-Box;">:~/</span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li></ul>
以下在Slave主机上配置
- 在slave主机上创建.ssh文件夹
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">mkdir ~/<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.ssh</span> // 修改权限 chmod <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">700</span> ~/<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.ssh</span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li></ul>
- 追加到授权文件”authorized_keys”
在master主机下进行测试
进行jdk,hadoop软件安装
jdk安装:
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">// 需要在真机中进入文件目录,地址为虚拟主机的ip地址 scp jdk-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">8</span>u45-linux-x64<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.tar</span><span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.gz</span> root<span class="hljs-localvars" style="Box-sizing: border-Box;">@192</span><span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.168</span><span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.122</span>:/usr/java scp jdk-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.125</span>:/usr/java scp jdk-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.124</span>:/usr/java</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li></ul>
- 使用以下指令进行加压
- 配置jdk环境变量
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vi /etc/profile </code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li></ul>
- 添加Java环境变量
- 使配置生效
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",102); Box-sizing: border-Box;">source</span> /etc/profile</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul>
- 验证安装成功
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">java <span class="hljs-attribute" style="Box-sizing: border-Box;">-version</span> <span class="hljs-comment" style="color: rgb(136,0); Box-sizing: border-Box;">// 如果出现以下信息,则配置成功</span> java version <span class="hljs-string" style="color: rgb(0,0); Box-sizing: border-Box;">"1.8.0_45"</span> Java(TM) SE Runtime Environment (build <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">1.8</span><span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.0</span>_45<span class="hljs-attribute" style="Box-sizing: border-Box;">-b14</span>) Java HotSpot(TM) <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">64</span><span class="hljs-attribute" style="Box-sizing: border-Box;">-Bit</span> Server VM (build <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">25.45</span><span class="hljs-attribute" style="Box-sizing: border-Box;">-b02</span>,mixed mode)</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li></ul>
Hadoop安装
- 使用以下指令将hadoop-2.7.0.tar.gz文件复制到/usr目录下
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">scp hadoop-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2.7</span>.<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span>.tar.gz root<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">122</span><span class="hljs-symbol" style="color: rgb(0,102); Box-sizing: border-Box;">:/usr/</span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul>
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">cd /usr tar zxvf hadoop-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2.7</span><span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.0</span><span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.gz</span> mv hadoop-<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.0</span> hadoop // 删除hadoop-<span class="hljs-number" style="color: rgb(0,68); Box-sizing: border-Box;">.gz</span>文件 rm –rf hadoop-<span class="hljs-number" style="color: rgb(0,68); Box-sizing: border-Box;">.gz</span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li></ul>
- 在”/usr/hadoop”下面创建tmp文件夹
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",102); Box-sizing: border-Box;">cd</span> /usr/hadoop mkdir tmp</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li></ul>
- 把Hadoop的安装路径添加到”/etc/profile”中
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vi /etc/profile // 将以下数据加入到文件末尾 <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">export</span> HADOOP_INSTALL=/usr/hadoop <span class="hljs-keyword" style="color: rgb(0,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span>/bin:<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span>/sbin<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${PATH}</span> <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">export</span> HADOOP_MAPRED_HOME=<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span> <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">export</span> HADOOP_COMMON_HOME=<span class="hljs-variable" style="color: rgb(102,136); Box-sizing: border-Box;">export</span> HADOOP_HDFS_HOME=<span class="hljs-variable" style="color: rgb(102,136); Box-sizing: border-Box;">export</span> YARN_HOME=<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALLL}</span> <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">export</span> HADOOP_COMMON_LIB_NATIVE_DIR=<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span>/lib/natvie <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">export</span> HADOOP_OPTS=<span class="hljs-string" style="color: rgb(0,0); Box-sizing: border-Box;">"-Djava.library.path=<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span>/lib:<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">${HADOOP_INSTALL}</span>/lib/native"</span> </code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li></ul>
- 重启”/etc/profile”
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li></ul>
配置hadoop(先只在Master主机配置,配置完后传入两个Slave主机)
- 设置hadoop-env.sh和yarn-env.sh中的java环境变量
5022792" class="prettyprint" name="code" style="white-space: nowrap; word-wrap: break-word; Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",102); Box-sizing: border-Box;">cd</span> /usr/hadoop/etc/hadoop/ vi hadoop-env.sh // 修改JAVA_HOME <span class="hljs-keyword" style="color: rgb(0,102); Box-sizing: border-Box;">8.0</span>_45</code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li></ul>
- 配置core-site.xml文件
- 配置hdfs-site.xml文件
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vi hdfs-site.xml // 修改文件内容为以下 <span class="hljs-tag" style="color: rgb(0,136);">configuration</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">property</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.namenode.name.dir<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>file:///usr/hadoop/dfs/name<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">property</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.datanode.data.dir<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>file:///usr/hadoop/dfs/data<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.replication<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>1<span class="hljs-tag" style="color: rgb(0,136);">property</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.nameservices<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>hadoop-cluster1<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">property</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.namenode.secondary.http-address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:50090<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>dfs.webhdfs.enabled<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0,136);">configuration</span>></span> </code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li></ul>
- 配置mapred-site.xml文件
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> vi mapred-site.xml // 修改文件为以下 <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>mapreduce.framework.name<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>yarn<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">final</span>></span>true<span class="hljs-tag" style="color: rgb(0,136);">final</span>></span> <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>mapreduce.jobtracker.http.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:50030<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>mapreduce.jobhistory.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:10020<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>mapreduce.jobhistory.webapp.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:19888<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>mapred.job.tracker<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>http://Master.Hadoop:9001<span class="hljs-tag" style="color: rgb(0,136);">property</span>></span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li></ul>
- 配置yarn-site.xml文件
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vi yarn-site.xml // 修改文件内容为以下 <span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.hostname<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>mapreduce_shuffle<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:8032<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.scheduler.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:8030<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.resource-tracker.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:8031<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.admin.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:8033<span class="hljs-tag" style="color: rgb(0,136);">name</span>></span>yarn.resourcemanager.webapp.address<span class="hljs-tag" style="color: rgb(0,136);">value</span>></span>Master.Hadoop:8088<span class="hljs-tag" style="color: rgb(0,136);">property</span>></span> </code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li><li style="Box-sizing: border-Box; padding: 0px 5px;">34</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li><li style="Box-sizing: border-Box; padding: 0px 5px;">34</li></ul>
配置Hadoop的集群
- 将Master中配置好的hadoop传入两个Slave中
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">scp -r /usr/hadoop root<span class="hljs-variable" style="color: rgb(102,102); Box-sizing: border-Box;">:/usr/</span> scp -r /usr/hadoop root<span class="hljs-variable" style="color: rgb(102,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li></ul>
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">cd /usr/hadoop/etc/hadoop vi slaves // 将文件内容修改为 Slave1<span class="hljs-preprocessor" style="color: rgb(68,68); Box-sizing: border-Box;">.Hadoop</span> Slave2<span class="hljs-preprocessor" style="color: rgb(68,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li></ul>
- 格式化HDFS文件系统
- 启动hadoop
- 验证hadoop
Box-sizing: border-Box; position: relative; overflow-y: hidden; overflow-x: auto; margin-top: 0px; margin-bottom: 1.1em; font-family: "Source Code Pro",0); Box-sizing: border-Box;">// 1. 直接在Master或Slave输入指令:</span> jps <span class="hljs-comment" style="color: rgb(136,0); Box-sizing: border-Box;">// 应该输出以下信息(端口号仅供参考)</span> Master: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">3930</span> ResourceManager <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">4506</span> Jps <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">3693</span> NameNode Slave: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2792</span> NodeManager <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2920</span> Jps <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2701</span> Datanode <span class="hljs-comment" style="color: rgb(136,0); Box-sizing: border-Box;">// 2. 输入以下指令</span> hadoop dfsadmin <span class="hljs-attribute" style="Box-sizing: border-Box;">-report</span> <span class="hljs-comment" style="color: rgb(136,0); Box-sizing: border-Box;">// 应该输出以下信息:</span> Configured Capacity: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">14382268416</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">13.39</span> GB) Present Capacity: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">10538565632</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">9.81</span> GB) DFS Remaining: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">10538557440</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">9.81</span> GB) DFS Used: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">8192</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">8</span> KB) DFS Used<span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span>: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0.00</span><span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span> Under replicated blocks: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> Blocks <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">with</span> corrupt replicas: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> Missing blocks: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> Missing blocks (<span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">with</span> replication factor <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">1</span>): <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> <span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">-------------------------------------------------</span> Live datanodes (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2</span>): Name: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.124</span>:<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">50010</span> (Slave2<span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">.</span>Hadoop) Hostname: Slave2<span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">.</span>Hadoop Decommission Status : Normal Configured Capacity: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">7191134208</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">6.70</span> GB) DFS Used: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">4096</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">4</span> KB) Non DFS Used: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">1921933312</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">1.79</span> GB) DFS Remaining: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">5269196800</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">4.91</span> GB) DFS Used<span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span> DFS Remaining<span class="hljs-subst" style="color: rgb(0,102); Box-sizing: border-Box;">73.27</span><span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span> Configured <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">Cache</span> Capacity: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">0</span> B) <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">Cache</span> Used: <span class="hljs-number" style="color: rgb(0,136); Box-sizing: border-Box;">Cache</span> Remaining: <span class="hljs-number" style="color: rgb(0,136); Box-sizing: border-Box;">Cache</span> Used<span class="hljs-subst" style="color: rgb(0,102); Box-sizing: border-Box;">100.00</span><span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span> <span class="hljs-keyword" style="color: rgb(0,136); Box-sizing: border-Box;">Cache</span> Remaining<span class="hljs-subst" style="color: rgb(0,0); Box-sizing: border-Box;">%</span> Xceivers: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">1</span> Last contact: Thu Jul <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">02</span> <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">10</span>:<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">45</span>:<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">04</span> CST <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">2015</span> Name: <span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">.125</span>:<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">50010</span> (Slave1<span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">.</span>Hadoop) Hostname: Slave1<span class="hljs-built_in" style="color: rgb(102,102); Box-sizing: border-Box;">1921769472</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">5269360640</span> (<span class="hljs-number" style="color: rgb(0,102); Box-sizing: border-Box;">73.28</span><span class="hljs-subst" style="color: rgb(0,102); Box-sizing: border-Box;">1</span></code><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li><li style="Box-sizing: border-Box; padding: 0px 5px;">34</li><li style="Box-sizing: border-Box; padding: 0px 5px;">35</li><li style="Box-sizing: border-Box; padding: 0px 5px;">36</li><li style="Box-sizing: border-Box; padding: 0px 5px;">37</li><li style="Box-sizing: border-Box; padding: 0px 5px;">38</li><li style="Box-sizing: border-Box; padding: 0px 5px;">39</li><li style="Box-sizing: border-Box; padding: 0px 5px;">40</li><li style="Box-sizing: border-Box; padding: 0px 5px;">41</li><li style="Box-sizing: border-Box; padding: 0px 5px;">42</li><li style="Box-sizing: border-Box; padding: 0px 5px;">43</li><li style="Box-sizing: border-Box; padding: 0px 5px;">44</li><li style="Box-sizing: border-Box; padding: 0px 5px;">45</li><li style="Box-sizing: border-Box; padding: 0px 5px;">46</li><li style="Box-sizing: border-Box; padding: 0px 5px;">47</li><li style="Box-sizing: border-Box; padding: 0px 5px;">48</li><li style="Box-sizing: border-Box; padding: 0px 5px;">49</li><li style="Box-sizing: border-Box; padding: 0px 5px;">50</li><li style="Box-sizing: border-Box; padding: 0px 5px;">51</li><li style="Box-sizing: border-Box; padding: 0px 5px;">52</li><li style="Box-sizing: border-Box; padding: 0px 5px;">53</li><li style="Box-sizing: border-Box; padding: 0px 5px;">54</li><li style="Box-sizing: border-Box; padding: 0px 5px;">55</li><li style="Box-sizing: border-Box; padding: 0px 5px;">56</li><li style="Box-sizing: border-Box; padding: 0px 5px;">57</li><li style="Box-sizing: border-Box; padding: 0px 5px;">58</li><li style="Box-sizing: border-Box; padding: 0px 5px;">59</li><li style="Box-sizing: border-Box; padding: 0px 5px;">60</li><li style="Box-sizing: border-Box; padding: 0px 5px;">61</li><li style="Box-sizing: border-Box; padding: 0px 5px;">62</li><li style="Box-sizing: border-Box; padding: 0px 5px;">63</li><li style="Box-sizing: border-Box; padding: 0px 5px;">64</li></ul><ul class="pre-numbering" style="Box-sizing: border-Box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221,238);"><li style="Box-sizing: border-Box; padding: 0px 5px;">1</li><li style="Box-sizing: border-Box; padding: 0px 5px;">2</li><li style="Box-sizing: border-Box; padding: 0px 5px;">3</li><li style="Box-sizing: border-Box; padding: 0px 5px;">4</li><li style="Box-sizing: border-Box; padding: 0px 5px;">5</li><li style="Box-sizing: border-Box; padding: 0px 5px;">6</li><li style="Box-sizing: border-Box; padding: 0px 5px;">7</li><li style="Box-sizing: border-Box; padding: 0px 5px;">8</li><li style="Box-sizing: border-Box; padding: 0px 5px;">9</li><li style="Box-sizing: border-Box; padding: 0px 5px;">10</li><li style="Box-sizing: border-Box; padding: 0px 5px;">11</li><li style="Box-sizing: border-Box; padding: 0px 5px;">12</li><li style="Box-sizing: border-Box; padding: 0px 5px;">13</li><li style="Box-sizing: border-Box; padding: 0px 5px;">14</li><li style="Box-sizing: border-Box; padding: 0px 5px;">15</li><li style="Box-sizing: border-Box; padding: 0px 5px;">16</li><li style="Box-sizing: border-Box; padding: 0px 5px;">17</li><li style="Box-sizing: border-Box; padding: 0px 5px;">18</li><li style="Box-sizing: border-Box; padding: 0px 5px;">19</li><li style="Box-sizing: border-Box; padding: 0px 5px;">20</li><li style="Box-sizing: border-Box; padding: 0px 5px;">21</li><li style="Box-sizing: border-Box; padding: 0px 5px;">22</li><li style="Box-sizing: border-Box; padding: 0px 5px;">23</li><li style="Box-sizing: border-Box; padding: 0px 5px;">24</li><li style="Box-sizing: border-Box; padding: 0px 5px;">25</li><li style="Box-sizing: border-Box; padding: 0px 5px;">26</li><li style="Box-sizing: border-Box; padding: 0px 5px;">27</li><li style="Box-sizing: border-Box; padding: 0px 5px;">28</li><li style="Box-sizing: border-Box; padding: 0px 5px;">29</li><li style="Box-sizing: border-Box; padding: 0px 5px;">30</li><li style="Box-sizing: border-Box; padding: 0px 5px;">31</li><li style="Box-sizing: border-Box; padding: 0px 5px;">32</li><li style="Box-sizing: border-Box; padding: 0px 5px;">33</li><li style="Box-sizing: border-Box; padding: 0px 5px;">34</li><li style="Box-sizing: border-Box; padding: 0px 5px;">35</li><li style="Box-sizing: border-Box; padding: 0px 5px;">36</li><li style="Box-sizing: border-Box; padding: 0px 5px;">37</li><li style="Box-sizing: border-Box; padding: 0px 5px;">38</li><li style="Box-sizing: border-Box; padding: 0px 5px;">39</li><li style="Box-sizing: border-Box; padding: 0px 5px;">40</li><li style="Box-sizing: border-Box; padding: 0px 5px;">41</li><li style="Box-sizing: border-Box; padding: 0px 5px;">42</li><li style="Box-sizing: border-Box; padding: 0px 5px;">43</li><li style="Box-sizing: border-Box; padding: 0px 5px;">44</li><li style="Box-sizing: border-Box; padding: 0px 5px;">45</li><li style="Box-sizing: border-Box; padding: 0px 5px;">46</li><li style="Box-sizing: border-Box; padding: 0px 5px;">47</li><li style="Box-sizing: border-Box; padding: 0px 5px;">48</li><li style="Box-sizing: border-Box; padding: 0px 5px;">49</li><li style="Box-sizing: border-Box; padding: 0px 5px;">50</li><li style="Box-sizing: border-Box; padding: 0px 5px;">51</li><li style="Box-sizing: border-Box; padding: 0px 5px;">52</li><li style="Box-sizing: border-Box; padding: 0px 5px;">53</li><li style="Box-sizing: border-Box; padding: 0px 5px;">54</li><li style="Box-sizing: border-Box; padding: 0px 5px;">55</li><li style="Box-sizing: border-Box; padding: 0px 5px;">56</li><li style="Box-sizing: border-Box; padding: 0px 5px;">57</li><li style="Box-sizing: border-Box; padding: 0px 5px;">58</li><li style="Box-sizing: border-Box; padding: 0px 5px;">59</li><li style="Box-sizing: border-Box; padding: 0px 5px;">60</li><li style="Box-sizing: border-Box; padding: 0px 5px;">61</li><li style="Box-sizing: border-Box; padding: 0px 5px;">62</li><li style="Box-sizing: border-Box; padding: 0px 5px;">63</li><li style="Box-sizing: border-Box; padding: 0px 5px;">64</li></ul><div class="save_code" style="Box-sizing: border-Box; position: absolute; width: 105px; height: 60px; right: 30px; top: 25px; color: rgb(255,255,255); cursor: pointer;"><img src="http://static.blog.csdn.net/images/save_snippets.png" style="border: none; Box-sizing: border-Box; max-width: 100%;" alt="" /></div>
- 访问网页
参考资料
- 在CentOS上安装Hadoop 2.x 集群
- centos安装配置hadoop超详细过程
- Centos下安装hadoop并与Eclipse连接
- Hadoop集群(第5期)_Hadoop安装配置
- 【分享】VirtualBox下安装CentOS 7 【图文教程】
- centos7没有安装ifconfig命令的解决方法
- [CentOS]安装软件:/lib/ld-linux.so.2: bad ELF interpreter解决
- CentOS6.4配置Hadoop-2.6.0集群配置安装指南(经过实战演练)【张振华.Jack】
- Hadoop “Unable to load native-hadoop library for your platform” error on CentOS
- CentOS静态IP修改网卡配置文件
- CentOS 6.5 配置hadoop 2.6.0伪分布式
- How can i use iptables on centos 7?