Ubuntu单机版 Storm 开发环境搭建与org.apache.storm.utils.NimbusLeaderNotFoundException问题

前端之家收集整理的这篇文章主要介绍了Ubuntu单机版 Storm 开发环境搭建与org.apache.storm.utils.NimbusLeaderNotFoundException问题前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

Ubuntu单机版 Storm 开发环境搭建


  • 查看Ubuntu
  • 安装JDK
  • 安装Pythod
  • 安装Zookeeper
  • 安装ZeroMQ
  • 安装Jzmp
  • 安装Storm

查看Ubuntu是32位还是64

uname -a

返回结果 ==> SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux,可知Ubuntu为64位



安装JDK

#下载64位的JDK

wget http://download.Oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz

#解压JDK

tar -zxf jdk-7u79-linux-x64.tar.gz

#设置JDK 的环境变量(以下步骤1、2、3、4)

#步骤1:打开profile文件 (/etc/profile)或者。bashrc 文件

我这里是 编辑.bahshrc 文件


执行命令

sudo gedit ~/.bashrc




Python 安装:

安装Python
#下载Python 
wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz
#解压python文件  
tar zxf Python-2.7.2.tgz
#进入python目录
cd Python-2.7.2
#编译和安装
./configure
make
make install

Zookeeper 安装:
安装Zookeeper

#下载zookeeper
wget http://apache.fayea.com/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.tar.gz

#解压zookeeper
tar zxf zookeeper-3.4.4.tar.gz

修改配置文件,设置环境变量。


ZeroMQ 安装:

#下载zeromq 我这里下载4.2.1

http://zeromq.org/intro:get-the-software

#编译和安装
./configure

#在configure时可能会报缺包,安装即可:sudo apt-get install g++ uuid-dev

make

sudo make install

sudo ldconfig

有些人可不能装一些软件 主要是因为sources.list 的问题:


先备份:

cp /etc/apt/sources.list /etc/apt/sources.list.bak 


安装 storm :

下载:

 https://storm.apache.org/downloads.html
安装Storm
#安装 storm 
#wget https://storm.apache.org/downloads.html/
#解压 storm
tar -zxvf  apache-storm-0.9.6.tar.gz
#设置环境变量
#步骤1:编辑 profile 目录
vim /etc/profile

配置配置文件:~/.bashrc


这里提供个相对好用的:

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-security main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-security main restricted universe multiverse

# 预发布软件源,不建议启用
# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse


JZMQ 安装:

我这里是下载了一个jzmq-master.zip 的包,然后按照如下操作 可以不要最后一个 mvn package

下载:https://github.com/zeromq/jzmq
(1) 把 jzmq-master.zip 拷贝到zeromq目录.
unzipjzmq-master.zip
(2)
cdjzmq-jni/
./autogen.sh
./configure
make
makeinstall
cd..
mvnpackage


配置文件配置如下:

#export JAVA_HOME=/usr/local/jdk1.8.0_121
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/bin:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

#Zookeeper
export ZOOKEEPER_HOME=/usr/local/zookeeper-3.3.6
export PATH=$PATH:$ZOOKEEPER_HOME/bin

# set hadoop environment
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL


#storm
export STORM_HOME=/usr/local/apache-storm-1.1.0
export PATH=$PATH:$STORM_HOME/bin


配置zk :

zk 安装目录下 conf 目录


zoo_sample.cfg 文件

cp zoo_sample.cfg zoo.cfg

这里配置一份单机版本的:


# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=10
# the directory where the snapshot is stored.
dataDir=/usr/local/zookeeper-3.3.6/data
# the port at which the clients will connect
clientPort=2181
#server.1=192.168.61.128


配置Storm :



storm.zookeeper.servers: 
      - "192.168.61.128"

#nimbus.host: "192.168.61.128"  
 nimbus.seeds: ["192.168.61.128"]
 
 ui.port: 9999
 storm.zookeeper.port: 2181
 storm.local.dir: "/usr/local/apache-storm-1.1.0/workdir"

 supervisor.slots.ports: 
       - 6700
       - 6701
       - 6702
       - 6703 


192.168.61.128 是本机虚拟机地址 ifconfig - a可看


但是 问题来了,

#步骤2:启动zookeeper
zkServer.sh start
zkServer.sh status  #查看zkserver是否成功启动

#启动storm
#步骤1:进入storm下的bin目录
cd /home/linux/apache-storm-0.9.6/bin
#启动服务
storm nimbus
storm supervisor
storm ui


启动之后一直 打开页面 一直报错:

org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts ["master" "slave1"]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?
    at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:90)
    at org.apache.storm.ui.core$all_topologies_summary.invoke(core.clj:434)
    at org.apache.storm.ui.core$fn__12605.invoke(core.clj:952)
    at org.apache.storm.shade.compojure.core$make_route$fn__4631.invoke(core.clj:100)
    at org.apache.storm.shade.compojure.core$if_route$fn__4619.invoke(core.clj:46)
    at org.apache.storm.shade.compojure.core$if_method$fn__4612.invoke(core.clj:31)
    at org.apache.storm.shade.compojure.core$routing$fn__4637.invoke(core.clj:113)
    at clojure.core$some.invoke(core.clj:2570)
    at org.apache.storm.shade.compojure.core$routing.doInvoke(core.clj:113)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invoke(core.clj:632)



网上方法: 1.采用集群安装方式

2. 删除 zk 、storm 节点,这里问题又来了,根本删不掉,还一直报错, 还是找不到 nimbus


增加解决: 发现是本机/etc/hosts 问题,这个不配置好, 启动不正常,一直困恼了好几天, 才搞定,网上的办法,一直说是delete 节点,但是实际操作根据delete 不了





必须配置好,重启 zk 重启 storm OK :


出现如下:

原文链接:https://www.f2er.com/ubuntu/352109.html

猜你在找的Ubuntu相关文章